Directory ownership lost in 2.9 and 2.10 when dir not empty

[Reuti]

Hi,

this was on the list quite some times as a bug in 2.9. I copy a  
directory tree as a result of a "find . -depth" and the ownerships  
are not restored when the directoyr is not empty. OTOH, I have to use  
-depth to retain the modifications dates of the directory. I hoped  
that it would be fixed in 2.10, but it's still there. For now I  
continue to use 2.8, which works fine.

I this no longer considered as a bug, and just the correct behavior  
in the future?

-- Reuti

[asc1000]

Reuti wrote:

Hi,

this was on the list quite some times as a bug in 2.9. I copy a  
directory tree as a result of a "find . -depth" and the ownerships  
are not restored when the directoyr is not empty. OTOH, I have to use  
-depth to retain the modifications dates of the directory. I hoped  
that it would be fixed in 2.10, but it's still there. For now I  
continue to use 2.8, which works fine.

I this no longer considered as a bug, and just the correct behavior  
in the future?

-- Reuti

Hello,

I have done some work to trace this problem.   Here is a way to reproduce it, an explanation of what is happening, and a workaround patch for 2.10.  Relevant files are attached.

makepath.c.diff
test.tar.gz

The simple test case directory tree:

test/
test/one/
test/two/
test/three/
test/three/file

Every directory (test, one, two, three) and the file (file) have modified date of oct 1.  I create an archive as follows:

    find test -depth | cpio -ov > test.cpio

Then I remove the original tree:

    rm -rf test

Now, I restore the tree:

    cat test.cpio | cpio -idumv

This results in all nodes except two getting their original oct 1 modify date.  The two exceptions are:

test/
test/three/

Those two nodes get a modify time set to current system clock.  The reason for this is because they have child nodes and are therefore created by make_path() which puts them into the delay_set_stat linked list.  After all data is restored, that list is processed by routines that do not have access to the original directory's ownership and atime and mtime, so that information gets destroyed and replaced by current system time, etc.

My question is, what is the reason for deferring that operation?  Can it be completed when the directory is created?

The workaround I'm using is to call apply_delayed_set_stat() right after delay_set_stat().  It is not elegant, but it makes the directory stats and timestamps come out correctly.  I do not really know if that adversely affects some other operational case, which is why I'd like this reviewed by someone more familiar with that code's intent.

Thanks,
Amardeep Chana
Indiana, US

[Tim Kientzle]

That delayed strategy is necessary to handle cases where
the parent directory is non-writable.

Try your test again, but change the permissions on the "test"
directory to read-only (chmod 555 test) before creating your
test.cpio archive.  I think you'll find that your patch
makes cpio unable to restore the archive.

I expect the real fix is to store ownership, atime,
and mtime in the delay list so that apply_delayed_set_stat()
can apply those as well.  (Might rename the routine to
apply_delayed_attributes() while you're making this change.)

Cheers,

Tim

asc1000 wrote:

>
> Reuti wrote:
>> Hi,
>>
>> this was on the list quite some times as a bug in 2.9. I copy a  
>> directory tree as a result of a "find . -depth" and the ownerships  
>> are not restored when the directoyr is not empty. OTOH, I have to use  
>> -depth to retain the modifications dates of the directory. I hoped  
>> that it would be fixed in 2.10, but it's still there. For now I  
>> continue to use 2.8, which works fine.
>>
>> I this no longer considered as a bug, and just the correct behavior  
>> in the future?
>>
>> -- Reuti
>>
>>
>
>
> Hello,
>
> I have done some work to trace this problem.   Here is a way to reproduce
> it, an explanation of what is happening, and a workaround patch for 2.10.
> Relevant files are attached.
>
> http://www.nabble.com/file/p26096793/makepath.c.diff makepath.c.diff
> http://www.nabble.com/file/p26096793/test.tar.gz test.tar.gz
>
> The simple test case directory tree:
>
> test/
> test/one/
> test/two/
> test/three/
> test/three/file
>
> Every directory (test, one, two, three) and the file (file) have modified
> date of oct 1.  I create an archive as follows:
>
>     find test -depth | cpio -ov > test.cpio
>
> Then I remove the original tree:
>
>     rm -rf test
>
> Now, I restore the tree:
>
>     cat test.cpio | cpio -idumv
>
> This results in all nodes except two getting their original oct 1 modify
> date.  The two exceptions are:
>
> test/
> test/three/
>
> Those two nodes get a modify time set to current system clock.  The reason
> for this is because they have child nodes and are therefore created by
> make_path() which puts them into the delay_set_stat linked list.  After all
> data is restored, that list is processed by routines that do not have access
> to the original directory's ownership and atime and mtime, so that
> information gets destroyed and replaced by current system time, etc.
>
> My question is, what is the reason for deferring that operation?  Can it be
> completed when the directory is created?
>
> The workaround I'm using is to call apply_delayed_set_stat() right after
> delay_set_stat().  It is not elegant, but it makes the directory stats and
> timestamps come out correctly.  I do not really know if that adversely
> affects some other operational case, which is why I'd like this reviewed by
> someone more familiar with that code's intent.
>
> Thanks,
> Amardeep Chana
> Indiana, US
>

[asc1000]

Tim Kientzle wrote:

That delayed strategy is necessary to handle cases where
the parent directory is non-writable.

Try your test again, but change the permissions on the "test"
directory to read-only (chmod 555 test) before creating your
test.cpio archive.  I think you'll find that your patch
makes cpio unable to restore the archive.

I expect the real fix is to store ownership, atime,
and mtime in the delay list so that apply_delayed_set_stat()
can apply those as well.  (Might rename the routine to
apply_delayed_attributes() while you're making this change.)

Cheers,

Tim

Hi Tim,

Thanks for clarifying the purpose.  I had suspected it was something along those lines after reading some previous messages related to this problem.  It didn't originally occur to me because I always use the -depth option to find, which puts the directory after its contents.  Incidentally, the chmod 555 test did not prevent the patched cpio from restoring the tree because the directory itself was processed after the contents.

I think I've come up with a solution that would handle all possible cases (directory item precedes contents, directory item follows contents, and directory item is not present at all).  It would involve generating the delayed list entries (including the properties as you suggest) from the directory item in the archive stream rather than from the implicit creation by make_path().  That way the restored directory will be created using umask thus guaranteeing the contents will be writable.  In the two cases where the directory entry is in the archive stream the permissions will be restored to original after all the files.  In the case where there is no directory item in the stream, they will remain as the default umask settings.

Perhaps that was the original intent!  I'll post a patch when it is ready.

Thanks,
Amardeep