tar | gzip problems

[cooch17]

Strange one - I have a large directory structure I'd like to tar up, preserving permissions and links and such. I wrote a small script (say, backup_files), which pipes tar to gzip to generate the .gz file.

cd /directory/of/interest
rm -r -f files.tar.gz
nohup tar cvpf -  . | gzip -9 > /directory/of/interest/files.tar.gz &


Seems to work fine, except that the .gz file is always reported as corrupted (have tried 3-4 different archive tools to open and access files - they all report that files.tar.gz is corrupt.


But...if I don't use a pipe

tar -cvpf files.tar *

then

gzip -9 files.tar

then the resulting files.tar.gz is absolutely fine. I've reproduced this sequence 3-4 times now.

So, something I can't suss out when tar pipes to gzip is causing a major problem.

Suggestions? Pointers to the obvious?

[Piotr Szlązak]

Hi!
Try using gzip -c:
nohup tar cvpf -  . | gzip -9 -c > /directory/of/interest/files.tar.gz &

Regards,
Piotrek

On Fri, Sep 25, 2009 at 16:10, cooch17 <cooch17@...> wrote:

>
> Strange one - I have a large directory structure I'd like to tar up,
> preserving permissions and links and such. I wrote a small script (say,
> backup_files), which pipes tar to gzip to generate the .gz file.
>
> cd /directory/of/interest
> rm -r -f files.tar.gz
> nohup tar cvpf -  . | gzip -9 > /directory/of/interest/files.tar.gz &
>
>
> Seems to work fine, except that the .gz file is always reported as corrupted
> (have tried 3-4 different archive tools to open and access files - they all
> report that files.tar.gz is corrupt.
>
>
> But...if I don't use a pipe
>
> tar -cvpf files.tar *
>
> then
>
> gzip -9 files.tar
>
> then the resulting files.tar.gz is absolutely fine. I've reproduced this
> sequence 3-4 times now.
>
> So, something I can't suss out when tar pipes to gzip is causing a major
> problem.
>
> Suggestions? Pointers to the obvious?
>
> --
> View this message in context: http://www.nabble.com/tar-%7C-gzip-problems-tp25612830p25612830.html
> Sent from the Gnu - Tar - Help mailing list archive at Nabble.com.

[Sergey Poznyakoff-2]

cooch17 <cooch17@...> ha escrit:

> nohup tar cvpf -  . | gzip -9 > /directory/of/interest/files.tar.gz &
>
> Seems to work fine, except that the .gz file is always reported as
> corrupted

I cannot reproduce this. What version of tar are you using?

Regards,
Sergey

[cooch17]

Sergey Poznyakoff-2 wrote:

cooch17 <[e-mail]> ha escrit:

> nohup tar cvpf -  . | gzip -9 > /directory/of/interest/files.tar.gz &
>
> Seems to work fine, except that the .gz file is always reported as
> corrupted

I cannot reproduce this. What version of tar are you using?

Regards,
Sergey

1.15.1 under CentOS 5.3.

[cooch17]

Piotr Szlązak wrote:

Hi!
Try using gzip -c:
nohup tar cvpf -  . | gzip -9 -c > /directory/of/interest/files.tar.gz &

Regards,
Piotrek

I ended up going with tar -cvpzf files.tar.gz *

which does the trick without any problems. I got a long answer on why my original code didn't work:


'piping' was never the problem.  The command below (tar cz...) also
uses pipes (you just don't see/type them).

The problem is taring up files (compressed or not) to a tarball *in the
same directory* as the files you are taring up and using '.' as your
source.  Using a '*' (shell wildcard) 'avoids' the problem, but adds
others (missed hidden files, etc.) -- it is *often* a mistake to use
shell wildcards with commands like tar (esp. a bare '*').  The proper
solution is either:

A) put your tarball someplace other than where you are tarring up:

tar czvf /tmp/files.tar.gz .

B) use the --exclude option to exclude the tarball being created:

tar czvf files.tar.gz . --exclude files.tar.gz