Looking for a quick script solution

[Stephen Goldman]

Hello Blu.
        I 'm faced with a directory with about 100 gzip files that need to be uncompressed.

        they are at s_2_0001_int.txt.p.gz and go to s_8_0098_nse.txt.p.gz


s_2_0001_int.txt.p.gz

s_8_0098_nse.txt.p.gz


        If there was ever a need for a script this is it.. little experience here though..

        I 'm thinking a for [i} something    
            to load the file  * gz
            then gunzip
            form 1 to 100

        not sure where to start.

        Open for some ideas

Thanks,
Stephen



     

             




s_2_0001_int.txt.p.gz

s_8_0098_nse.txt.p.gz




Stephen Goldman
System Administrator
MIT Biology
sgoldman@...
_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Derek Atkins]

How about (in bash):

foreach f in *.gz ; do
  gunzip $f
done

-derek

Quoting sgoldman <sgoldman@...>:

> Hello Blu.
>        I 'm faced with a directory with about 100 gzip files that
> need to be uncompressed.
>
>        they are at s_2_0001_int.txt.p.gz and go to s_8_0098_nse.txt.p.gz
>
>
> s_2_0001_int.txt.p.gz
>
> s_8_0098_nse.txt.p.gz
>
>
>        If there was ever a need for a script this is it.. little
> experience here though..
>
>        I 'm thinking a for [i} something
>            to load the file  * gz
>            then gunzip
>            form 1 to 100
>
>        not sure where to start.
>
>        Open for some ideas
>
> Thanks,
> Stephen
>
>
>
>
>
>
>
>
>
>
> s_2_0001_int.txt.p.gz
>
> s_8_0098_nse.txt.p.gz
>
>
>
>
> Stephen Goldman
> System Administrator
> MIT Biology
> sgoldman@...
> _______________________________________________
> Discuss mailing list
> Discuss@...
> http://lists.blu.org/mailman/listinfo/discuss
>

--
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord@...                        PGP key available

_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Rajiv Aaron Manglani-2]

>        I 'm faced with a directory with about 100 gzip files that  
> need to be uncompressed.
>         they are at s_2_0001_int.txt.p.gz and go to  
> s_8_0098_nse.txt.p.gz
>         Open for some ideas

gunzip *


_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Richard Pieri]

On Jul 25, 2009, at 8:29 PM, sgoldman wrote:
>        If there was ever a need for a script this is it.. little  
> experience here though..


No script needed.  Just:

   gunzip *.gz

If that throws a too many files error then you should use a find  
command:

   find . -depth 1 -name "*.gz" -print | xargs gunzip

This finds all files in the current directory (.) but no deeper than  
the current directory (-depth 1) that match "*.gz" and execs gunzip on  
each one.  It is slower than a single shell glob.  Much slower.  It  
has to open and stat the directories and then exec gunzip for every  
file.  On the other hand, it works even when you have so many files  
that you can't use a shell glob.

--Rich P.
_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Gordon Marx]

On Thu, Jun 25, 2009 at 10:33 PM, Richard Pieri<richard.pieri@...> wrote:
> This finds all files in the current directory (.) but no deeper than
> the current directory (-depth 1) that match "*.gz" and execs gunzip on
> each one.

That's actually the behavior of find's -exec option. xargs is smarter
than that -- it executes command lines that are as near as possible to
the maximum length each time.

Gordon
_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Stephen Goldman]

THANKS !


gunzip  *.gz



Stephen Goldman
System Administrator
MIT Biology
sgoldman@...

----- Original Message -----
From: "sgoldman" <sgoldman@...>
To: <discuss@...>
Sent: Saturday, July 25, 2009 8:29 PM
Subject: Looking for a quick script solution

> Hello Blu.
>        I 'm faced with a directory with about 100 gzip files that need to
> be uncompressed.
>
>        they are at s_2_0001_int.txt.p.gz and go to s_8_0098_nse.txt.p.gz
>
>
> s_2_0001_int.txt.p.gz
>
> s_8_0098_nse.txt.p.gz
>
>
>        If there was ever a need for a script this is it.. little
> experience here though..
>
>        I 'm thinking a for [i} something
>            to load the file  * gz
>            then gunzip
>            form 1 to 100
>
>        not sure where to start.
>
>        Open for some ideas
>
> Thanks,
> Stephen
>
>
>
>
>
>
>
>
>
>
> s_2_0001_int.txt.p.gz
>
> s_8_0098_nse.txt.p.gz
>
>
>
>
> Stephen Goldman
> System Administrator
> MIT Biology
> sgoldman@...
> _______________________________________________
> Discuss mailing list
> Discuss@...
> http://lists.blu.org/mailman/listinfo/discuss
>

_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Richard Pieri]

On Jun 25, 2009, at 11:03 PM, Gordon Marx wrote:
> That's actually the behavior of find's -exec option. xargs is smarter
> than that -- it executes command lines that are as near as possible to
> the maximum length each time.


You are correct.  I had forgotten.  I knew there was a reason for  
using xargs rather than, say:

   find . -name "*.gz" -exec gunzip {} \;

--Rich P.

_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Tom Metro-16]

Gordon Marx wrote:
> Richard Pieri wrote:
>> This finds all files in the current directory...and execs gunzip on
>> each one.
>
> That's actually the behavior of find's -exec option. xargs is smarter
> than that -- it executes command lines that are as near as possible to
> the maximum length each time.

I've noticed that newer versions of find actually support an extension
to -exec that emulates xargs (from the GNU find version 4.2.32 man page;
note the trailing "+"):

   -exec command {} +
     This  variant  of  the -exec option runs the specified command on
     the selected files, but the command line is  built  by  appending
     each  selected  file name at the end; the total number of invoca-
     tions of the command will be much less than the number of matched
     files.  The command line is built in much the same way that xargs
     builds its command lines.  Only one instance of {}  is   allowed
     within  the  command.   The  command  is executed in the starting
     directory.

The same man page also recommends using -execdir over -exec, as there
"are unavoidable security problems surrounding use of the -exec option,"
and that -execdir "avoids race conditions during resolution of the paths
to the matched files." Although it seems to open up another hole: "If
you use this option, you must ensure that your $PATH environment
variable does not reference the current directory; otherwise, an
attacker can run any commands they like by leaving an
appropriately-named file in a directory in which you will run -execdir."

-execdir uses the same syntax, but does a CWD to the directory
containing the file being acted on before performing the operation. But
note that the batching performed by the "+" extension would be limited
to files in a single directory, so if your files happen to be widely
dispersed in a file system, you'll get a lot of separate exec() calls.

  -Tom

--
Tom Metro
Venture Logic, Newton, MA, USA
"Enterprise solutions through open source."
Professional Profile: http://tmetro.venturelogic.com/
_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Richard Pieri]

On Jun 26, 2009, at 1:43 AM, Tom Metro wrote:
> I've noticed that newer versions of find actually support an extension
> to -exec that emulates xargs (from the GNU find version 4.2.32 man  
> page;
> note the trailing "+"):

FreeBSD 7's (I think) find also includes this feature.

As a rule I try to avoid using non-standard, dare I say proprietary,  
extensions to tools I frequently use.  They aren't portable across  
different operating systems.

--Rich P.

_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss

[Tom McLaughlin-2]

On Fri, Jun 26, 2009 at 10:52 AM, Richard Pieri<richard.pieri@...> wrote:
> On Jun 26, 2009, at 1:43 AM, Tom Metro wrote:
>> I've noticed that newer versions of find actually support an extension
>> to -exec that emulates xargs (from the GNU find version 4.2.32 man
>> page;
>> note the trailing "+"):
>
> FreeBSD 7's (I think) find also includes this feature.

`find` in FreeBSD 7 implements many GNUisms not previously available.

--
| tmclaugh at sdf.lonestar.org                 tmclaugh at FreeBSD.org |
| FreeBSD                                       http://www.FreeBSD.org |
_______________________________________________
Discuss mailing list
Discuss@...
http://lists.blu.org/mailman/listinfo/discuss