Ant find duplicate files

[cvsusr]

Hi

In my ant build,

I need to find duplicate files under set of dirs and if found, copy it to the temp directory.

say in my current dir i have chk/src/a.txt and chk/src2/a.txt and chk/src2/b.txt

the task should find a.txt under src and src2 and copy it to chk/test dir with folder names

Can this  be done in any way..

Thanks

[Matt Benson-2]

With stock Ant, I think you'd have to do this in two passes, but the  
<present> selector (see http://ant.apache.org/manual/CoreTypes/ 
selectors.html#presentselect) should help.

-Matt

On Nov 6, 2009, at 3:45 PM, cvsusr wrote:

>
> Hi
>
> In my ant build,
>
> I need to find duplicate files under set of dirs and if found, copy  
> it to
> the temp directory.
>
> say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
> chk/src2/b.txt
>
> the task should find a.txt under src and src2 and copy it to chk/
> test dir
> with folder names
>
> Can this  be done in any way..
>
> Thanks
> --
> View this message in context: http://old.nabble.com/Ant-find- 
> duplicate-files-tp26230907p26230907.html
> Sent from the Ant - Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@...
> For additional commands, e-mail: user-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Francis Galiegue-4]

On Fri, Nov 6, 2009 at 22:45, cvsusr <spers@...> wrote:

>
> Hi
>
> In my ant build,
>
> I need to find duplicate files under set of dirs and if found, copy it to
> the temp directory.
>
> say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
> chk/src2/b.txt
>
> the task should find a.txt under src and src2 and copy it to chk/test dir
> with folder names
>
> Can this  be done in any way..
>

You know, there's no need to do a double post... Well, anyway, as you
still don't have a workable answer so far, here is the solution that I
envisioned. Quoting your original mail:

> need to find duplicate files under set of dirs and if found, copy it to
> the temp directory.
>
> say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
> chk/src2/b.txt
>
> the task should find a.txt under src and src2 and copy it to chk/test dir
> with folder names
>
> Can this  be done in any way..

What do you call "folder names"? Give an example, please. Until then,
the following is assumed here:

* that since src/a.txt exists but not src/b.txt, what you want as a
result is chk/test/a.txt, but not chk/test/b.txt;
* that there's only one level deep (ie, there won't be src/a/b.txt and
src2/a/b.txt).

If this is not the case, the below example will NOT work, and you'll
have to be more precise about what you really want. I have worked with
much more complicated schemes than that wrt filenames, and have found
ant-contrib to be of outstanding value for such cases. Base ant just
cannot do it that easily, or in such a convoluted way that trying to
get it to work is a nightmare. Honestly.

Anyway, here is what I'd do in your case:

<property name="srcdir" value="chk/src2"/>

<fileset id="origfiles" dir="${srcdir}" includes="*.txt"/>

<target name="doit">
    <for param="origfile">
        <path>
            <fileset refid="origfiles"/>
        </path>
        <sequential>
            <propertyregex value="dup" override="yes"
input="@{origfile}" regexp="/src2/" replace="/src/"/>
            <if>
                <available file="${dup}" type="file"/>
                <then>
                    <!-- Yes, mkdir each time. It's not that much of a
waste, really, since <mkdir> will silently ignore an existing
directory -->
                    <mkdir dir="chk/test"/>
                    <copy file="@{origfile}" todir="chk/test"/>
                </then>
            </if>
        </sequential>
    </for>
</target>

This uses three tasks from ant-contrib: for, if/then, propertyregex.
And it's only a simple usage: <propertyregex> is ant's Swiss army
knife for regexes and beats every other regex mechanism in ant.

--
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : [phone]
Tel : [phone]
fge@...
40 avenue Raymond Poincaré
75116 Paris

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[cvsusr]

Thanks for ur replies.. i will look in to that .. before that i shall give you the explanation of the question.


I have dirs like

Title/t1
title/t2
title/t3

and more sub dirs in each t1/ t2 /t3//

but at one level we have one folder named "src" under all t1, t2, t3 folder and we do have files in it with same name as a.txt with other files


structure

Title
|-------------t1/dir/src-------------a.txt
                                -------------foo.c
                                -------------c.txt
|-------------t2/src ---------------a.txt
                             ---------------c.txt
                             ---------------something.xml
|-------------t3/c/a/src/ ---|
                                        |------b.txt
                                        | --------c.txt


My real task is to find all the repeated file in the the subdirectories, if its present, copy them to the temp directory with folder name mappings. Since a.txt is present in t1 and t2 folder.. i copy them to a temp folder and hence both are same file, i do a build picking a.txt file from the t1 or t2 but then copying the build artifact to the t1 and t2 again.. if atall there are any changes need to be done in a.txt, developers can go to the temp folder and does the change in one place either t1 or t2 folder under my temp dir. the i do a build of that and copy back to the respective folders.. in this case t1 and t2..

this is the whole idea.. hope its clear..

to achieve this scenario.. i splitted m tasks and my first part is to find the duplicate or repeated file list and store it in a temp dir and then go on to next steps..

can you pls.. let me know a solution for this..

thanks
sri

cvsusr wrote:

Hi

In my ant build,

I need to find duplicate files under set of dirs and if found, copy it to the temp directory.

say in my current dir i have chk/src/a.txt and chk/src2/a.txt and chk/src2/b.txt

the task should find a.txt under src and src2 and copy it to chk/test dir with folder names

Can this  be done in any way..

Thanks

[cvsusr]

Few more clarifications----


to be more clear... i can take one copy of the repeated files under the temp directory and do a build but need to keep track of the dirs where these files are repeated.. so that i could copy back my build artifact to the respective folders.. in the above case..

im finding the repeated a.txt under title
found it under t1 and t2..
copy once copy either from t1 or t2 to temp dir
keep track of the folders where and all the a.txt is repeated
do the build in temp dir and copy back the build output to the respective folder as per our earlier track.. in this case, copy back the build artifact to t1 and t2.


---------------------------


Thanks for ur replies.. i will look in to that .. before that i shall give you the explanation of the question.


I have dirs like

Title/t1
title/t2
title/t3

and more sub dirs in each t1/ t2 /t3//

but at one level we have one folder named "src" under all t1, t2, t3 folder and we do have files in it with same name as a.txt with other files


structure

Title
|-------------t1/dir/src-------------a.txt
                                -------------foo.c
                                -------------c.txt
|-------------t2/src ---------------a.txt
                             ---------------c.txt
                             ---------------something.xml
|-------------t3/c/a/src/ ---|
                                        |------b.txt
                                        | --------c.txt


My real task is to find all the repeated file in the the subdirectories, if its present, copy them to the temp directory with folder name mappings. Since a.txt is present in t1 and t2 folder.. i copy them to a temp folder and hence both are same file, i do a build picking a.txt file from the t1 or t2 but then copying the build artifact to the t1 and t2 again.. if atall there are any changes need to be done in a.txt, developers can go to the temp folder and does the change in one place either t1 or t2 folder under my temp dir. the i do a build of that and copy back to the respective folders.. in this case t1 and t2..

this is the whole idea.. hope its clear..

to achieve this scenario.. i splitted m tasks and my first part is to find the duplicate or repeated file list and store it in a temp dir and then go on to next steps..

can you pls.. let me know a solution for this..

thanks
sri

cvsusr wrote:

Hi

In my ant build,

I need to find duplicate files under set of dirs and if found, copy it to the temp directory.

say in my current dir i have chk/src/a.txt and chk/src2/a.txt and chk/src2/b.txt

the task should find a.txt under src and src2 and copy it to chk/test dir with folder names

Can this  be done in any way..

Thanks

[cvsusr]

Hi

Can anyone help me in this.. im trying various ways but not able to achieve a solution..

I used <preset task to check for the same files existing in two dirs.. its working fine when we know the target dir to compare and only when we need to compare two dirs..

In my case i need to check all the dirs available and pickup the repeated files..

Is there anyway to achieve this?? request ur help.

thanks.

Thanks for ur replies.. i will look in to that .. before that i shall give you the explanation of the question.


I have dirs like

Title/t1
title/t2
title/t3

and more sub dirs in each t1/ t2 /t3//

but at one level we have one folder named "src" under all t1, t2, t3 folder and we do have files in it with same name as a.txt with other files


structure

Title
|-------------t1/dir/src-------------a.txt
                                -------------foo.c
                                -------------c.txt
|-------------t2/src ---------------a.txt
                             ---------------c.txt
                             ---------------something.xml
|-------------t3/c/a/src/ ---|
                                        |------b.txt
                                        | --------c.txt


My real task is to find all the repeated file in the the subdirectories, if its present, copy them to the temp directory with folder name mappings. Since a.txt is present in t1 and t2 folder.. i copy them to a temp folder and hence both are same file, i do a build picking a.txt file from the t1 or t2 but then copying the build artifact to the t1 and t2 again.. if atall there are any changes need to be done in a.txt, developers can go to the temp folder and does the change in one place either t1 or t2 folder under my temp dir. the i do a build of that and copy back to the respective folders.. in this case t1 and t2..

this is the whole idea.. hope its clear..

to achieve this scenario.. i splitted m tasks and my first part is to find the duplicate or repeated file list and store it in a temp dir and then go on to next steps..

can you pls.. let me know a solution for this..

thanks
sri

cvsusr wrote:

Hi

In my ant build,

I need to find duplicate files under set of dirs and if found, copy it to the temp directory.

say in my current dir i have chk/src/a.txt and chk/src2/a.txt and chk/src2/b.txt

the task should find a.txt under src and src2 and copy it to chk/test dir with folder names

Can this  be done in any way..

Thanks

[cvsusr]

its <present> task not preset..  


Hi

Can anyone help me in this.. im trying various ways but not able to achieve a solution..

I used <preset task to check for the same files existing in two dirs.. its working fine when we know the target dir to compare and only when we need to compare two dirs..

In my case i need to check all the dirs available and pickup the repeated files..

Is there anyway to achieve this?? request ur help.

thanks.

cvsusr wrote:

Thanks for ur replies.. i will look in to that .. before that i shall give you the explanation of the question.


I have dirs like

Title/t1
title/t2
title/t3

and more sub dirs in each t1/ t2 /t3//

but at one level we have one folder named "src" under all t1, t2, t3 folder and we do have files in it with same name as a.txt with other files


structure

Title
|-------------t1/dir/src-------------a.txt
                                -------------foo.c
                                -------------c.txt
|-------------t2/src ---------------a.txt
                             ---------------c.txt
                             ---------------something.xml
|-------------t3/c/a/src/ ---|
                                        |------b.txt
                                        | --------c.txt


My real task is to find all the repeated file in the the subdirectories, if its present, copy them to the temp directory with folder name mappings. Since a.txt is present in t1 and t2 folder.. i copy them to a temp folder and hence both are same file, i do a build picking a.txt file from the t1 or t2 but then copying the build artifact to the t1 and t2 again.. if atall there are any changes need to be done in a.txt, developers can go to the temp folder and does the change in one place either t1 or t2 folder under my temp dir. the i do a build of that and copy back to the respective folders.. in this case t1 and t2..

this is the whole idea.. hope its clear..

to achieve this scenario.. i splitted m tasks and my first part is to find the duplicate or repeated file list and store it in a temp dir and then go on to next steps..

can you pls.. let me know a solution for this..

thanks
sri

cvsusr wrote:

Hi

In my ant build,

I need to find duplicate files under set of dirs and if found, copy it to the temp directory.

say in my current dir i have chk/src/a.txt and chk/src2/a.txt and chk/src2/b.txt

the task should find a.txt under src and src2 and copy it to chk/test dir with folder names

Can this  be done in any way..

Thanks

[cvsusr]

Is there any Ant experts who can provide a solutions for this scenario...  ??

Thanks in advance..

cvsusr wrote:

Hi

In my ant build,

I need to find duplicate files under set of dirs and if found, copy it to the temp directory.

say in my current dir i have chk/src/a.txt and chk/src2/a.txt and chk/src2/b.txt

the task should find a.txt under src and src2 and copy it to chk/test dir with folder names

Can this  be done in any way..

Thanks

[Adam Leggett (UPCO)]

If you are comfortable with scripting using syntax akin to Java, i'd
suggest you use the groovy ant task to do something like this

http://groovy.codehaus.org/The+groovy+Ant+Task


Thanks

Adam

On Wed, 2009-11-11 at 02:13 -0800, cvsusr wrote:

> Is there any Ant experts who can provide a solutions for this scenario...  ??
>
> Thanks in advance..
>
>
>
>
> cvsusr wrote:
> >
> > Hi
> >
> > In my ant build,
> >
> > I need to find duplicate files under set of dirs and if found, copy it to
> > the temp directory.
> >
> > say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
> > chk/src2/b.txt
> >
> > the task should find a.txt under src and src2 and copy it to chk/test dir
> > with folder names
> >
> > Can this  be done in any way..
> >
> > Thanks
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[cuberoot]

Adam's suggestion of a script is probably going to be the simplest solution.

Another option may be to take a look at the Checksum task and generate MD5
hashes for the files, then compare the MD5 hash values for each file, if the
values are equal, then you know you have a duplicate. You could then delete
all the MD5 files.

This solution may be a bit clunky, but it should work.


On Wed, Nov 11, 2009 at 10:18 AM, Adam Leggett (UPCO) <
adam.leggett@...> wrote:

> If you are comfortable with scripting using syntax akin to Java, i'd
> suggest you use the groovy ant task to do something like this
>
> http://groovy.codehaus.org/The+groovy+Ant+Task
>
>
> Thanks
>
> Adam
>
> On Wed, 2009-11-11 at 02:13 -0800, cvsusr wrote:
> > Is there any Ant experts who can provide a solutions for this scenario...
>  ??
> >
> > Thanks in advance..
> >
> >
> >
> >
> > cvsusr wrote:
> > >
> > > Hi
> > >
> > > In my ant build,
> > >
> > > I need to find duplicate files under set of dirs and if found, copy it
> to
> > > the temp directory.
> > >
> > > say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
> > > chk/src2/b.txt
> > >
> > > the task should find a.txt under src and src2 and copy it to chk/test
> dir
> > > with folder names
> > >
> > > Can this  be done in any way..
> > >
> > > Thanks
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@...
> For additional commands, e-mail: user-help@...
>
>

[cvsusr]

I really need to learn scripting and do it.. Can I used java script?? I will give a try..

Is it so that i need to write java script and call it using groovy task??

Adam Leggett (UPCO) wrote:

If you are comfortable with scripting using syntax akin to Java, i'd
suggest you use the groovy ant task to do something like this

http://groovy.codehaus.org/The+groovy+Ant+Task


Thanks

Adam

On Wed, 2009-11-11 at 02:13 -0800, cvsusr wrote:
> Is there any Ant experts who can provide a solutions for this scenario...  ??
>
> Thanks in advance..
>
>
>
>
> cvsusr wrote:
> >
> > Hi
> >
> > In my ant build,
> >
> > I need to find duplicate files under set of dirs and if found, copy it to
> > the temp directory.
> >
> > say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
> > chk/src2/b.txt
> >
> > the task should find a.txt under src and src2 and copy it to chk/test dir
> > with folder names
> >
> > Can this  be done in any way..
> >
> > Thanks
> >
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[Adam Leggett (UPCO)]

Ok, if your preference is javascript you could try the optional
scriptdef task instead:

http://ant.apache.org/manual/OptionalTasks/scriptdef.html

The above page provides an example of how to iterate through filesets
which should give you a good start.

Thanks

Adam

On Wed, 2009-11-11 at 02:28 -0800, cvsusr wrote:

> I really need to learn scripting and do it.. Can I used java script?? I will
> give a try..
>
> Is it so that i need to write java script and call it using groovy task??
>
>
> Adam Leggett (UPCO) wrote:
> >
> > If you are comfortable with scripting using syntax akin to Java, i'd
> > suggest you use the groovy ant task to do something like this
> >
> > http://groovy.codehaus.org/The+groovy+Ant+Task
> >
> >
> > Thanks
> >
> > Adam
> >
> > On Wed, 2009-11-11 at 02:13 -0800, cvsusr wrote:
> >> Is there any Ant experts who can provide a solutions for this scenario...
> >> ??
> >>
> >> Thanks in advance..
> >>
> >>
> >>
> >>
> >> cvsusr wrote:
> >> >
> >> > Hi
> >> >
> >> > In my ant build,
> >> >
> >> > I need to find duplicate files under set of dirs and if found, copy it
> >> to
> >> > the temp directory.
> >> >
> >> > say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
> >> > chk/src2/b.txt
> >> >
> >> > the task should find a.txt under src and src2 and copy it to chk/test
> >> dir
> >> > with folder names
> >> >
> >> > Can this  be done in any way..
> >> >
> >> > Thanks
> >> >
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@...
> > For additional commands, e-mail: user-help@...
> >
> >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Scot P. Floess-2]

For scripting.... I like beanshell myself :)

On Wed, 11 Nov 2009, cvsusr wrote:

>
> Is there any Ant experts who can provide a solutions for this scenario...  ??
>
> Thanks in advance..
>
>
>
>
> cvsusr wrote:
>>
>> Hi
>>
>> In my ant build,
>>
>> I need to find duplicate files under set of dirs and if found, copy it to
>> the temp directory.
>>
>> say in my current dir i have chk/src/a.txt and chk/src2/a.txt and
>> chk/src2/b.txt
>>
>> the task should find a.txt under src and src2 and copy it to chk/test dir
>> with folder names
>>
>> Can this  be done in any way..
>>
>> Thanks
>>
>
> --
> View this message in context: http://old.nabble.com/Ant-find-duplicate-files-tp26230907p26298938.html
> Sent from the Ant - Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@...
> For additional commands, e-mail: user-help@...
>
>

Scot P. Floess
27 Lake Royale
Louisburg, NC  27549

252-478-8087 (Home)
919-890-8117 (Work)

Chief Architect JPlate   http://sourceforge.net/projects/jplate
Chief Architect JavaPIM  http://sourceforge.net/projects/javapim

Architect Keros          http://sourceforge.net/projects/keros

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...