Why does the <fileset> implementation keep track of filesNotIncluded, dirsNotIncluded?

[Bryan Pendleton]

I ran what I thought was a fairly simple Ant target, to
remove unwanted JUnit output from my build tree:

     <target name="cleanTests">
         <delete>
             <fileset dir="${SRCROOT}" includes="**/TEST-*.txt"/>
         </delete>
     </target>

I found that running this target uses memory proportional
to the size of my entire tree, rather than using memory
proportional to the number of JUnit test output files in my
tree, which is what I expected.

I got a memory dump of my Ant process when it ran out of memory,
and looked inside, and discovered that all the memory was consumed
by the 'filesNotIncluded' and 'dirsNotIncluded' Vector objects
in the DirectoryScanner class.

I don't understand why DirectoryScanner feels the need to track
the files and directories that were *not* included in my fileset.

How would I use such information in my Ant build?

Could this be made optional, so that DirectoryScanner only tracks
this information if I ask it to?

thanks,

bryan


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Stefan Bodewig]

On 2009-10-05, Bryan Pendleton <bpendleton@...> wrote:

> I don't understand why DirectoryScanner feels the need to track
> the files and directories that were *not* included in my fileset.

The DirectoryScanner API provides methods to access those files and
directories after a scan.  Right now I don't think any task uses them,
but they have been part of the initial API early 2000 and been kept
around (and functional) for backwards compatibility reasons.

> Could this be made optional, so that DirectoryScanner only tracks
> this information if I ask it to?

In a way it already does.  I.e. when you have a pattern like

<include name="foo/**"/>

Ant will only traverse into foo and ignore all sibling directories of
foo because they can not contain included files/dirs.  When it is asked
for excluded or not-included files it will recognize it hasn't scanned
everything and start a new scan in "slow" mode.

The code is really optimized for speed rather than memory usage
therefore information on files and directories that have already been
seen once is kept.  Otherwise Ant would have to start a completely fresh
scan when asked for not-included or excluded files.

From the top of my head I'm not sure whether you could modify
DirectoryScanner's behaviour without being forced to modify subclasses
as well.  Ant alone has a few of them - for zip and tar archives,
DependScanner for the depend task and one for the remote directories in
the ftp task - and it could mean breaking the contract for those
subclasses provided by third parties.

> I ran what I thought was a fairly simple Ant target, to
> remove unwanted JUnit output from my build tree:

>     <target name="cleanTests">
>         <delete>
>             <fileset dir="${SRCROOT}" includes="**/TEST-*.txt"/>
>         </delete>
>     </target>

Is there any way you can make the pattern more specific or can the test
output files really be everywhere in you source tree?

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Bryan Pendleton]

Stefan Bodewig wrote:
> Is there any way you can make the pattern more specific or can the test
> output files really be everywhere in you source tree?

Thanks for the reply, Stefan!

A more specific inclusion pattern indeed helps, and we'll use that solution.

bryan



---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...