Have gnutar pass --no-name to gzip

[Michael D. Adams]

Is there some way to convince gnutar that it should pass "--no-name"
(or the C API equivalent) to gzip?  (This prevents gzip from embedding
the current time in the file [1].)

The reason I ask is that I need to pass "--no-name" to gzip (so
re-tar-ing the same file twice produces identical binary output).  I
could write it as a pipeline (e.g. "tar -cf - ... | gzip --no-name
>file.tar.gz"), but there are two problems with that.  First, gnutar
may[2] be more efficient at running gzip internally (e.g. if it uses a
library rather than spawning a gzip process).  Since this is in an
inner loop [3], I want to run things in the most efficient way
possible.  Second, I need to be able to detect any error codes coming
from tar but POSIX shells only report errors in the last command of
the pipeline.

Michael D. Adams

[1] I've also considered using other compression flags (e.g. -j for
bzip2 or -Z for compress) since they don't embed the current time in
their files, but they have other draw backs.  "bzip2" is just too
sloooow.  "compress" exits with error code 2 if the compressed file
would be larger than the original.  This is not a good thing to happen
in the middle of this particular script as catching that error
condition and recovering from it makes things difficult.

[2] Timing tests that I've run are inconclusive.

[3] Yes, I'm invoking tar hundreds of thousands of times which isn't
exactly efficient, but I need separate tar files in the output and so
I don't see any way around that part of the inefficiency.  Some
inefficiencies can't be avoided, but I'd like to avoid them where ever
possible.

[Tim Kientzle]

Look at the gzip man page, specifically the GZIP environment variable.
You can use this to pass arbitrary command-line flags to the gzip program.

Note that gnu tar does simply open a pipe to gzip, so there should be no
performance difference between gtar -z and an explicit shell pipeline.

Tim

Michael D. Adams wrote:

> Is there some way to convince gnutar that it should pass "--no-name"
> (or the C API equivalent) to gzip?  (This prevents gzip from embedding
> the current time in the file [1].)
>
> The reason I ask is that I need to pass "--no-name" to gzip (so
> re-tar-ing the same file twice produces identical binary output).  I
> could write it as a pipeline (e.g. "tar -cf - ... | gzip --no-name
>> file.tar.gz"), but there are two problems with that.  First, gnutar
> may[2] be more efficient at running gzip internally (e.g. if it uses a
> library rather than spawning a gzip process).  Since this is in an
> inner loop [3], I want to run things in the most efficient way
> possible.  Second, I need to be able to detect any error codes coming
> from tar but POSIX shells only report errors in the last command of
> the pipeline.
>
> Michael D. Adams
>
> [1] I've also considered using other compression flags (e.g. -j for
> bzip2 or -Z for compress) since they don't embed the current time in
> their files, but they have other draw backs.  "bzip2" is just too
> sloooow.  "compress" exits with error code 2 if the compressed file
> would be larger than the original.  This is not a good thing to happen
> in the middle of this particular script as catching that error
> condition and recovering from it makes things difficult.
>
> [2] Timing tests that I've run are inconclusive.
>
> [3] Yes, I'm invoking tar hundreds of thousands of times which isn't
> exactly efficient, but I need separate tar files in the output and so
> I don't see any way around that part of the inefficiency.  Some
> inefficiencies can't be avoided, but I'd like to avoid them where ever
> possible.
>
>
>
>

[Michael D. Adams]

Awesome! That fits my needs exactly.  Thank you!

On Sun, Oct 18, 2009 at 4:42 PM, Tim Kientzle <kientzle@...> wrote:

> Look at the gzip man page, specifically the GZIP environment variable. You
> can use this to pass arbitrary command-line flags to the gzip program.
>
> Note that gnu tar does simply open a pipe to gzip, so there should be no
> performance difference between gtar -z and an explicit shell pipeline.
>
> Tim
>
> Michael D. Adams wrote:
>>
>> Is there some way to convince gnutar that it should pass "--no-name"
>> (or the C API equivalent) to gzip?  (This prevents gzip from embedding
>> the current time in the file [1].)
>>
>> The reason I ask is that I need to pass "--no-name" to gzip (so
>> re-tar-ing the same file twice produces identical binary output).  I
>> could write it as a pipeline (e.g. "tar -cf - ... | gzip --no-name
>>>
>>> file.tar.gz"), but there are two problems with that.  First, gnutar
>>
>> may[2] be more efficient at running gzip internally (e.g. if it uses a
>> library rather than spawning a gzip process).  Since this is in an
>> inner loop [3], I want to run things in the most efficient way
>> possible.  Second, I need to be able to detect any error codes coming
>> from tar but POSIX shells only report errors in the last command of
>> the pipeline.
>>
>> Michael D. Adams
>>
>> [1] I've also considered using other compression flags (e.g. -j for
>> bzip2 or -Z for compress) since they don't embed the current time in
>> their files, but they have other draw backs.  "bzip2" is just too
>> sloooow.  "compress" exits with error code 2 if the compressed file
>> would be larger than the original.  This is not a good thing to happen
>> in the middle of this particular script as catching that error
>> condition and recovering from it makes things difficult.
>>
>> [2] Timing tests that I've run are inconclusive.
>>
>> [3] Yes, I'm invoking tar hundreds of thousands of times which isn't
>> exactly efficient, but I need separate tar files in the output and so
>> I don't see any way around that part of the inefficiency.  Some
>> inefficiencies can't be avoided, but I'd like to avoid them where ever
>> possible.
>>
>>
>>
>>
>