some performance regressions (and fixes)

[Adam Murdoch-2]

Hi,

I've just committed fixes for some performance regressions which have
snuck in since the 0.7 release:

- Ported AnnotationProcessingTaskFactory from Groovy to Java, and added
some caching.

- Use ASM instead of Groovy to generate task subclasses which mix in
convention mapping and dynamic properties.

- Cache all scripts. In particular, cache empty/missing init scripts and
the default buildSrc script. More on this below.

Using our performance test multi-project build (which has 25 projects),
gradle -t executes in:

Gradle 0.8-20090829161512+1000:  8.41 seconds (this is the version we're
using for gradlew)
Gradle 0.7:  3.98 seconds
Gradle head:  3.58 seconds

So, head is a now a little faster than the 0.7 release.

Also, the developerBuild is down from ~30 min to ~20 min on my
not-particularly-good laptop.

One implication of caching every script is that we don't always have a
directory in which to put the .gradle cache. So, I've changed the script
compilation to cache all scripts under ~/.gradle/scriptCache. An
advantage of this is we no longer end up with .gradle directories
scattered all through the source tree (unless you end up using an
internal repository). The downside is we will need some way to garbage
collect this cache. This isn't actually a new problem, because we needed
to solve this any way - It's just more important now.


Adam


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Adam Murdoch-2]

Some other performance numbers, doing a full build of the performance
test multi-project build.

Gradle head:
  gradle clean build, 22.26 seconds

Gradle 0.7
  gradle clean jar, 24.12 seconds

Maven 2.2:
  mvn clean package surefire-report:report, 37.29 seconds

Ant 1.7.0:
   ant clean jar, 15.97 seconds


Adam


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[John Murph]

On Tue, Sep 1, 2009 at 7:44 PM, Adam Murdoch <a@...> wrote:

I've changed the script compilation to cache all scripts under ~/.gradle/scriptCache. An advantage of this is we no longer end up with .gradle directories scattered all through the source tree (unless you end up using an internal repository). The downside is we will need some way to garbage collect this cache. This isn't actually a new problem, because we needed to solve this any way - It's just more important now.

I like have a single place for the cache (although I was thinking about putting the cache is the root project directory).  I wonder if Gradle might not want to always have a "clean" task (even if not using a plugin) that would clean the ~/.gradle stuff that is used for the current project.  The Java plugin could then make it's "clean" task depend on this default task (or maybe not... I know we've talked about having clean delete the script caches because they can get out of sync and cause problems, but maybe that's just a bug that needs to get fixed).  I don't really like Gradle always defining a task, but this seems more like a task than a command line option.


--
John Murph
Automated Logic Research Team

[hdockter]

On Sep 2, 2009, at 3:32 AM, Adam Murdoch wrote:

>
> Some other performance numbers, doing a full build of the  
> performance test multi-project build.
>
> Gradle head:
> gradle clean build, 22.26 seconds
>
> Gradle 0.7
> gradle clean jar, 24.12 seconds
>
> Maven 2.2:
> mvn clean package surefire-report:report, 37.29 seconds
>
> Ant 1.7.0:
>  ant clean jar, 15.97 seconds

Excellent.

Another area where we should be able to improve the performance are  
our Ivy resolves. At the moment for example a resolve of testCompile  
does not seem to benefit from an earlier resolve of compile. If I do a  
similar resolve with an Ant script, ivy performs much better. I  
couldn't figure out yet how to improve this. I have send an email to  
the Ivy list asking for help.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Adam Murdoch-2]

John Murph wrote:

On Tue, Sep 1, 2009 at 7:44 PM, Adam Murdoch <a@...> wrote:

I've changed the script compilation to cache all scripts under ~/.gradle/scriptCache. An advantage of this is we no longer end up with .gradle directories scattered all through the source tree (unless you end up using an internal repository). The downside is we will need some way to garbage collect this cache. This isn't actually a new problem, because we needed to solve this any way - It's just more important now.

I like have a single place for the cache (although I was thinking about putting the cache is the root project directory).

This would be a good option. Unfortunately, we can't cache the scripts until we know where the root project dir is, which is a bit late for init and settings scripts.

  I wonder if Gradle might not want to always have a "clean" task (even if not using a plugin) that would clean the ~/.gradle stuff that is used for the current project.  The Java plugin could then make it's "clean" task depend on this default task (or maybe not... I know we've talked about having clean delete the script caches because they can get out of sync and cause problems, but maybe that's just a bug that needs to get fixed).  I don't really like Gradle always defining a task, but this seems more like a task than a command line option.

The idea of using a task to clean up the caches is a good one. I would use something other than 'clean'. I really don't want Gradle to recompile my build scripts, or re-download dependencies, or re-hash my source files, or re-calculating source file dependencies after I've done a 'clean', because these things are inputs to my build, and aren't actually affected by deleting my build output.


Adam

[hdockter]

<snip>

> Another area where we should be able to improve the performance are  
> our Ivy resolves. At the moment for example a resolve of testCompile  
> does not seem to benefit from an earlier resolve of compile. If I do  
> a similar resolve with an Ant script, ivy performs much better. I  
> couldn't figure out yet how to improve this. I have send an email to  
> the Ivy list asking for help.

We have found the reasons for this difference but haven't fixed Gradle  
yet to make use of these Ivy optimizations.

For the record: Some interesting information about Ivy performance can  
be found here: https://issues.apache.org/jira/browse/IVY-872

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

On Sep 3, 2009, at 6:50 AM, Adam Murdoch wrote:

<snip>

>
> The idea of using a task to clean up the caches is a good one.

http://jira.codehaus.org/browse/GRADLE-625

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:

> Hi,
>
> I've just committed fixes for some performance regressions which  
> have snuck in since the 0.7 release:
>
> - Ported AnnotationProcessingTaskFactory from Groovy to Java, and  
> added some caching.
>
> - Use ASM instead of Groovy to generate task subclasses which mix in  
> convention mapping and dynamic properties.
>
> - Cache all scripts. In particular, cache empty/missing init scripts  
> and the default buildSrc script. More on this below.
>
> Using our performance test multi-project build (which has 25  
> projects), gradle -t executes in:
>
> Gradle 0.8-20090829161512+1000:  8.41 seconds (this is the version  
> we're using for gradlew)
> Gradle 0.7:  3.98 seconds
> Gradle head:  3.58 seconds
>
> So, head is a now a little faster than the 0.7 release.
>
> Also, the developerBuild is down from ~30 min to ~20 min on my not-
> particularly-good laptop.
>
> One implication of caching every script is that we don't always have  
> a directory in which to put the .gradle cache. So, I've changed the  
> script compilation to cache all scripts under ~/.gradle/scriptCache.  
> An advantage of this is we no longer end up with .gradle directories  
> scattered all through the source tree (unless you end up using an  
> internal repository). The downside is we will need some way to  
> garbage collect this cache. This isn't actually a new problem,  
> because we needed to solve this any way - It's just more important  
> now.
>

There are other areas where we also need a place to store metadata. In  
those cases we don't have the situation of possibly not having a  
directory.

- Changedetection (at the moment stored in <checkedDir>/.gradle)

Should we move this also to ~/.gradle ?

- task history (build/.gradle/task-execution-history)

I thought it makes sense to put the task history in the build dir. If  
the output is deleted you might as well delete the history (which  
purpose is to control which output needs to be reproduced). We avoid  
any GC/stale problems that way.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Adam Murdoch-2]

Hans Dockter wrote:

>
> On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:
>
>> Hi,
>>
>> I've just committed fixes for some performance regressions which have
>> snuck in since the 0.7 release:
>>
>> - Ported AnnotationProcessingTaskFactory from Groovy to Java, and
>> added some caching.
>>
>> - Use ASM instead of Groovy to generate task subclasses which mix in
>> convention mapping and dynamic properties.
>>
>> - Cache all scripts. In particular, cache empty/missing init scripts
>> and the default buildSrc script. More on this below.
>>
>> Using our performance test multi-project build (which has 25
>> projects), gradle -t executes in:
>>
>> Gradle 0.8-20090829161512+1000:  8.41 seconds (this is the version
>> we're using for gradlew)
>> Gradle 0.7:  3.98 seconds
>> Gradle head:  3.58 seconds
>>
>> So, head is a now a little faster than the 0.7 release.
>>
>> Also, the developerBuild is down from ~30 min to ~20 min on my
>> not-particularly-good laptop.
>>
>> One implication of caching every script is that we don't always have
>> a directory in which to put the .gradle cache. So, I've changed the
>> script compilation to cache all scripts under ~/.gradle/scriptCache.
>> An advantage of this is we no longer end up with .gradle directories
>> scattered all through the source tree (unless you end up using an
>> internal repository). The downside is we will need some way to
>> garbage collect this cache. This isn't actually a new problem,
>> because we needed to solve this any way - It's just more important now.
>>
>
> There are other areas where we also need a place to store metadata. In
> those cases we don't have the situation of possibly not having a
> directory.
>
> - Changedetection (at the moment stored in <checkedDir>/.gradle)
>
> Should we move this also to ~/.gradle ?
>

I think we should cache everything in the same place. But that's really
just my personal preference.

> - task history (build/.gradle/task-execution-history)
>
> I thought it makes sense to put the task history in the build dir. If
> the output is deleted you might as well delete the history (which
> purpose is to control which output needs to be reproduced). We avoid
> any GC/stale problems that way.
>

Except it's not quite right. It assumes all tasks which generate output
do so into the build directory. This means that after a clean, we'll
incorrectly re-execute tasks which generate their output somewhere else,
or which do not generate any output.

The rule for skipping a task is: skip the task if 1) its input files
have changed since last time it successfully executed, and 2) its output
files exist.

It's predicate 2) which deals with rebuilding after a clean, not
predicate 1). This means there's no reason to store this information in
the build directory.


Adam



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

On Sep 9, 2009, at 1:25 AM, Adam Murdoch wrote:

>
>
> Hans Dockter wrote:
>>
>> On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:
>>
>>> Hi,
>>>
>>> I've just committed fixes for some performance regressions which  
>>> have snuck in since the 0.7 release:
>>>
>>> - Ported AnnotationProcessingTaskFactory from Groovy to Java, and  
>>> added some caching.
>>>
>>> - Use ASM instead of Groovy to generate task subclasses which mix  
>>> in convention mapping and dynamic properties.
>>>
>>> - Cache all scripts. In particular, cache empty/missing init  
>>> scripts and the default buildSrc script. More on this below.
>>>
>>> Using our performance test multi-project build (which has 25  
>>> projects), gradle -t executes in:
>>>
>>> Gradle 0.8-20090829161512+1000:  8.41 seconds (this is the version  
>>> we're using for gradlew)
>>> Gradle 0.7:  3.98 seconds
>>> Gradle head:  3.58 seconds
>>>
>>> So, head is a now a little faster than the 0.7 release.
>>>
>>> Also, the developerBuild is down from ~30 min to ~20 min on my not-
>>> particularly-good laptop.
>>>
>>> One implication of caching every script is that we don't always  
>>> have a directory in which to put the .gradle cache. So, I've  
>>> changed the script compilation to cache all scripts under  
>>> ~/.gradle/scriptCache. An advantage of this is we no longer end up  
>>> with .gradle directories scattered all through the source tree  
>>> (unless you end up using an internal repository). The downside is  
>>> we will need some way to garbage collect this cache. This isn't  
>>> actually a new problem, because we needed to solve this any way -  
>>> It's just more important now.
>>>
>>
>> There are other areas where we also need a place to store metadata.  
>> In those cases we don't have the situation of possibly not having a  
>> directory.
>>
>> - Changedetection (at the moment stored in <checkedDir>/.gradle)
>>
>> Should we move this also to ~/.gradle ?
>>
>
> I think we should cache everything in the same place. But that's  
> really just my personal preference.
>
>> - task history (build/.gradle/task-execution-history)
>>
>> I thought it makes sense to put the task history in the build dir.  
>> If the output is deleted you might as well delete the history  
>> (which purpose is to control which output needs to be reproduced).  
>> We avoid any GC/stale problems that way.
>>
>
> Except it's not quite right.

I know. We talked about it a little in our last dev conference with  
Steve and Tom.

> It assumes all tasks which generate output do so into the build  
> directory. This means that after a clean, we'll incorrectly re-
> execute tasks which generate their output somewhere else,

Yes

> or which do not generate any output.

Task with no output don't have a history in the current implementation  
as the history is part of the output handling. The history returns if  
the output was created successfully and the last time the output was  
modified. That makes no sense for let's say the clean task. This  
behavior is also owned to the fact that we use the build directory for  
storing the history (which collides with a clean history). But the  
major reason is that right now our only history use case if for output  
generation optimization.

>
> The rule for skipping a task is: skip the task if 1) its input files  
> have changed since last time it successfully executed, and 2) its  
> output files exist.

That would be the most reliable approach and what we should do that in  
the future. Stuff would work out of the box. But for 0.8 we will go  
with custom onlyIf rules I think.

They would look like:

test.onlyIf { !test.output.wasSuccessfullyCreated ||  
test.output.lastModified < testCompile.output.lastModified ... }

That means if someone manually deletes build output the build would  
behave in an erroneous way.

>
> It's predicate 2) which deals with rebuilding after a clean, not  
> predicate 1). This means there's no reason to store this information  
> in the build directory.

No question about that. But I guess this is 0.9 stuff, isn't it?

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

On Sep 9, 2009, at 8:27 AM, Hans Dockter wrote:

>
> On Sep 9, 2009, at 1:25 AM, Adam Murdoch wrote:
>
>>
>>
>> Hans Dockter wrote:
>>>
>>> On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:
>>>
>>>> Hi,
>>>>
>>>> I've just committed fixes for some performance regressions which  
>>>> have snuck in since the 0.7 release:
>>>>
>>>> - Ported AnnotationProcessingTaskFactory from Groovy to Java, and  
>>>> added some caching.
>>>>
>>>> - Use ASM instead of Groovy to generate task subclasses which mix  
>>>> in convention mapping and dynamic properties.
>>>>
>>>> - Cache all scripts. In particular, cache empty/missing init  
>>>> scripts and the default buildSrc script. More on this below.
>>>>
>>>> Using our performance test multi-project build (which has 25  
>>>> projects), gradle -t executes in:
>>>>
>>>> Gradle 0.8-20090829161512+1000:  8.41 seconds (this is the  
>>>> version we're using for gradlew)
>>>> Gradle 0.7:  3.98 seconds
>>>> Gradle head:  3.58 seconds
>>>>
>>>> So, head is a now a little faster than the 0.7 release.
>>>>
>>>> Also, the developerBuild is down from ~30 min to ~20 min on my  
>>>> not-particularly-good laptop.
>>>>
>>>> One implication of caching every script is that we don't always  
>>>> have a directory in which to put the .gradle cache. So, I've  
>>>> changed the script compilation to cache all scripts under  
>>>> ~/.gradle/scriptCache. An advantage of this is we no longer end  
>>>> up with .gradle directories scattered all through the source tree  
>>>> (unless you end up using an internal repository). The downside is  
>>>> we will need some way to garbage collect this cache. This isn't  
>>>> actually a new problem, because we needed to solve this any way -  
>>>> It's just more important now.
>>>>
>>>
>>> There are other areas where we also need a place to store  
>>> metadata. In those cases we don't have the situation of possibly  
>>> not having a directory.
>>>
>>> - Changedetection (at the moment stored in <checkedDir>/.gradle)
>>>
>>> Should we move this also to ~/.gradle ?
>>>
>>
>> I think we should cache everything in the same place. But that's  
>> really just my personal preference.
>>
>>> - task history (build/.gradle/task-execution-history)
>>>
>>> I thought it makes sense to put the task history in the build dir.  
>>> If the output is deleted you might as well delete the history  
>>> (which purpose is to control which output needs to be reproduced).  
>>> We avoid any GC/stale problems that way.
>>>
>>
>> Except it's not quite right.
>
> I know. We talked about it a little in our last dev conference with  
> Steve and Tom.
>
>> It assumes all tasks which generate output do so into the build  
>> directory. This means that after a clean, we'll incorrectly re-
>> execute tasks which generate their output somewhere else,
>
> Yes
>
>> or which do not generate any output.
>
> Task with no output don't have a history in the current  
> implementation as the history is part of the output handling. The  
> history returns if the output was created successfully and the last  
> time the output was modified. That makes no sense for let's say the  
> clean task. This behavior is also owned to the fact that we use the  
> build directory for storing the history (which collides with a clean  
> history). But the major reason is that right now our only history  
> use case if for output generation optimization.
>
>>
>> The rule for skipping a task is: skip the task if 1) its input  
>> files have changed since last time it successfully executed, and 2)  
>> its output files exist.
>
> That would be the most reliable approach and what we should do that  
> in the future. Stuff would work out of the box. But for 0.8 we will  
> go with custom onlyIf rules I think.
>
> They would look like:
>
> test.onlyIf { !test.output.wasSuccessfullyCreated ||  
> test.output.lastModified < testCompile.output.lastModified ... }
>
> That means if someone manually deletes build output the build would  
> behave in an erroneous way.

Of course we could check already for output in such custom onlyIf's.  
What would be the check for example for the test task? Check if some  
test-results exists, check if all necessary test-results exists? I can  
imagine that the output checks could become a bit time consuming. So  
relying only on the input and that the last execution was successful  
is a possible performance optimization but not as reliable.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Tom Eyckmans]

2009/9/9 Hans Dockter <mail@...>

On Sep 9, 2009, at 8:27 AM, Hans Dockter wrote:

On Sep 9, 2009, at 1:25 AM, Adam Murdoch wrote:

Hans Dockter wrote:

On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:

Hi,

I've just committed fixes for some performance regressions which have snuck in since the 0.7 release:

- Ported AnnotationProcessingTaskFactory from Groovy to Java, and added some caching.

- Use ASM instead of Groovy to generate task subclasses which mix in convention mapping and dynamic properties.

- Cache all scripts. In particular, cache empty/missing init scripts and the default buildSrc script. More on this below.

Using our performance test multi-project build (which has 25 projects), gradle -t executes in:

Gradle 0.8-20090829161512+1000:  8.41 seconds (this is the version we're using for gradlew)
Gradle 0.7:  3.98 seconds
Gradle head:  3.58 seconds

So, head is a now a little faster than the 0.7 release.

Also, the developerBuild is down from ~30 min to ~20 min on my not-particularly-good laptop.

One implication of caching every script is that we don't always have a directory in which to put the .gradle cache. So, I've changed the script compilation to cache all scripts under ~/.gradle/scriptCache. An advantage of this is we no longer end up with .gradle directories scattered all through the source tree (unless you end up using an internal repository). The downside is we will need some way to garbage collect this cache. This isn't actually a new problem, because we needed to solve this any way - It's just more important now.

There are other areas where we also need a place to store metadata. In those cases we don't have the situation of possibly not having a directory.

- Changedetection (at the moment stored in <checkedDir>/.gradle)

Should we move this also to ~/.gradle ?

I think we should cache everything in the same place. But that's really just my personal preference.

This doesn't seem right to me if you store the state of a directory that is used by multiple projects you will have always have one project not acting on new/changed/deleted files, so this should really be in the project .gradle directory.

- task history (build/.gradle/task-execution-history)

I thought it makes sense to put the task history in the build dir. If the output is deleted you might as well delete the history (which purpose is to control which output needs to be reproduced). We avoid any GC/stale problems that way.

Except it's not quite right.

I know. We talked about it a little in our last dev conference with Steve and Tom.

It assumes all tasks which generate output do so into the build directory. This means that after a clean, we'll incorrectly re-execute tasks which generate their output somewhere else,

Yes

or which do not generate any output.

Task with no output don't have a history in the current implementation as the history is part of the output handling. The history returns if the output was created successfully and the last time the output was modified. That makes no sense for let's say the clean task. This behavior is also owned to the fact that we use the build directory for storing the history (which collides with a clean history). But the major reason is that right now our only history use case if for output generation optimization.

The rule for skipping a task is: skip the task if 1) its input files have changed since last time it successfully executed, and 2) its output files exist.

That would be the most reliable approach and what we should do that in the future. Stuff would work out of the box. But for 0.8 we will go with custom onlyIf rules I think.

They would look like:

test.onlyIf { !test.output.wasSuccessfullyCreated || test.output.lastModified < testCompile.output.lastModified ... }

That means if someone manually deletes build output the build would behave in an erroneous way.

Of course we could check already for output in such custom onlyIf's. What would be the check for example for the test task? Check if some test-results exists, check if all necessary test-results exists? I can imagine that the output checks could become a bit time consuming. So relying only on the input and that the last execution was successful is a possible performance optimization but not as reliable.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email

[Adam Murdoch-2]

Tom Eyckmans wrote:

2009/9/9 Hans Dockter <mail@...>

On Sep 9, 2009, at 8:27 AM, Hans Dockter wrote:

On Sep 9, 2009, at 1:25 AM, Adam Murdoch wrote:

Hans Dockter wrote:

On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:

Hi,

I've just committed fixes for some performance regressions which have snuck in since the 0.7 release:

- Ported AnnotationProcessingTaskFactory from Groovy to Java, and added some caching.

- Use ASM instead of Groovy to generate task subclasses which mix in convention mapping and dynamic properties.

- Cache all scripts. In particular, cache empty/missing init scripts and the default buildSrc script. More on this below.

Using our performance test multi-project build (which has 25 projects), gradle -t executes in:

Gradle 0.8-20090829161512+1000:  8.41 seconds (this is the version we're using for gradlew)
Gradle 0.7:  3.98 seconds
Gradle head:  3.58 seconds

So, head is a now a little faster than the 0.7 release.

Also, the developerBuild is down from ~30 min to ~20 min on my not-particularly-good laptop.

One implication of caching every script is that we don't always have a directory in which to put the .gradle cache. So, I've changed the script compilation to cache all scripts under ~/.gradle/scriptCache. An advantage of this is we no longer end up with .gradle directories scattered all through the source tree (unless you end up using an internal repository). The downside is we will need some way to garbage collect this cache. This isn't actually a new problem, because we needed to solve this any way - It's just more important now.

There are other areas where we also need a place to store metadata. In those cases we don't have the situation of possibly not having a directory.

- Changedetection (at the moment stored in <checkedDir>/.gradle)

Should we move this also to ~/.gradle ?

I think we should cache everything in the same place. But that's really just my personal preference.

This doesn't seem right to me if you store the state of a directory that is used by multiple projects you will have always have one project not acting on new/changed/deleted files, so this should really be in the project .gradle directory.

Then you'd have exactly the same problem if that project is included in multiple builds.

What I meant was that I'd like us to store all our state under ~/.gradle somewhere. It would be separated by build or project as appropriate under that directory.

Some benefits of storing this stuff under $userHomeDir:

- We deal with read-only source trees.

- All the state for a build is under one directory heirarchy. This makes it easy to garbage collect. Right now we store some stuff in $userHomeDir, some in $rootDir/.gradle, some in $projectDir/build.

- We have a command-line option to specify $userHomeDir, so that a user or a CI build can point this to whichever location they prefer. Which might be $rootDir/.gradle. They can also do the same in an init script.

- We don't litter .gradle dirs through the source tree. I find this quite annoying, and I don't imagine I'm alone.

Actually, these are really the benefits of having a single state directory heirarchy per build. It could be $userHomeDir, or it could be $rootDir/.gradle


Adam

[hdockter]

>
>
> - We don't litter .gradle dirs through the source tree. I find this  
> quite annoying, and I don't imagine I'm alone.

Minor point to support this. I really enjoy that GitHub is not  
polluting every source dir like svn does. I can easily copy stuff  
around.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Russel Winder-4]

On Fri, 2009-09-18 at 10:19 +0200, Hans Dockter wrote:
> >
> >
> > - We don't litter .gradle dirs through the source tree. I find this  
> > quite annoying, and I don't imagine I'm alone.
>
> Minor point to support this. I really enjoy that GitHub is not  
> polluting every source dir like svn does. I can easily copy stuff  
> around.

This is a Git, Bazaar, Mercurial feature, not a GitHub feature.  Apart
from the fact that DVCS is just better than CVCS, not having extra
(e.g. .svn) directories scattered around a working tree is the a hugely
important thing -- it means running scripts over the codebase is far,
far easier.

--
Russel.
=============================================================================
Dr Russel Winder      Partner
                                            xmpp: russel@...
Concertant LLP        t: [phone], [phone]
41 Buckmaster Road,   f: [phone]   voip: sip:russel.winder@...
London SW11 1EN, UK   m: [phone]   skype: russel_winder

[hdockter]

On Sep 18, 2009, at 10:29 AM, Russel Winder wrote:

> On Fri, 2009-09-18 at 10:19 +0200, Hans Dockter wrote:
>>>
>>>
>>> - We don't litter .gradle dirs through the source tree. I find this
>>> quite annoying, and I don't imagine I'm alone.
>>
>> Minor point to support this. I really enjoy that GitHub is not
>> polluting every source dir like svn does. I can easily copy stuff
>> around.
>
> This is a Git, Bazaar, Mercurial feature, not a GitHub feature.

Even I know this ;). Sorry for the typo.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Russel Winder-4]

On Fri, 2009-09-18 at 11:00 +0200, Hans Dockter wrote:

> On Sep 18, 2009, at 10:29 AM, Russel Winder wrote:
>
> > On Fri, 2009-09-18 at 10:19 +0200, Hans Dockter wrote:
> >>>
> >>>
> >>> - We don't litter .gradle dirs through the source tree. I find this
> >>> quite annoying, and I don't imagine I'm alone.
> >>
> >> Minor point to support this. I really enjoy that GitHub is not
> >> polluting every source dir like svn does. I can easily copy stuff
> >> around.
> >
> > This is a Git, Bazaar, Mercurial feature, not a GitHub feature.
>
> Even I know this ;). Sorry for the typo.

No problem, I'm just making sure everyone's awake ;-)

Real irritation of the moment is that versions of dependencies of Gradle
are now getting scattered around the place, some required but . . .
Also many of the version of dependencies used in Gradle are now really
very old.  Should there be a programme of moving to latest versions of
everything.  And should this be a continuous activity or something done
once a week, once a month, once a quarter, once a year, once a
decade, . . .

However this drifts far from the original topic.

--
Russel.
=============================================================================
Dr Russel Winder      Partner
                                            xmpp: russel@...
Concertant LLP        t: [phone], [phone]
41 Buckmaster Road,   f: [phone]   voip: sip:russel.winder@...
London SW11 1EN, UK   m: [phone]   skype: russel_winder

[hdockter]

On Sep 18, 2009, at 11:07 AM, Russel Winder wrote:

> On Fri, 2009-09-18 at 11:00 +0200, Hans Dockter wrote:
>> On Sep 18, 2009, at 10:29 AM, Russel Winder wrote:
>>
>>> On Fri, 2009-09-18 at 10:19 +0200, Hans Dockter wrote:
>>>>>
>>>>>
>>>>> - We don't litter .gradle dirs through the source tree. I find  
>>>>> this
>>>>> quite annoying, and I don't imagine I'm alone.
>>>>
>>>> Minor point to support this. I really enjoy that GitHub is not
>>>> polluting every source dir like svn does. I can easily copy stuff
>>>> around.
>>>
>>> This is a Git, Bazaar, Mercurial feature, not a GitHub feature.
>>
>> Even I know this ;). Sorry for the typo.
>
> No problem, I'm just making sure everyone's awake ;-)
>
> Real irritation of the moment is that versions of dependencies of  
> Gradle
> are now getting scattered around the place, some required but . . .
> Also many of the version of dependencies used in Gradle are now really
> very old.

Which ones do you have in mind?

> Should there be a programme of moving to latest versions of
> everything.  And should this be a continuous activity or something  
> done
> once a week, once a month, once a quarter, once a year, once a
> decade, . . .

It should be a continued manual activity rather at the beginning of a  
new release stream.

I have created a Wiki page for this: http://docs.codehaus.org/display/GRADLE/Libs+Gradle+Core

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Adam Murdoch-2]

Russel Winder wrote:

On Fri, 2009-09-18 at 11:00 +0200, Hans Dockter wrote:
  

On Sep 18, 2009, at 10:29 AM, Russel Winder wrote:

    

On Fri, 2009-09-18 at 10:19 +0200, Hans Dockter wrote:
      

- We don't litter .gradle dirs through the source tree. I find this
quite annoying, and I don't imagine I'm alone.
          

Minor point to support this. I really enjoy that GitHub is not
polluting every source dir like svn does. I can easily copy stuff
around.
        

This is a Git, Bazaar, Mercurial feature, not a GitHub feature.
      

Even I know this ;). Sorry for the typo.
    

No problem, I'm just making sure everyone's awake ;-)

Real irritation of the moment is that versions of dependencies of Gradle
are now getting scattered around the place, some required but . . .
Also many of the version of dependencies used in Gradle are now really
very old.  Should there be a programme of moving to latest versions of
everything.  And should this be a continuous activity or something done
once a week, once a month, once a quarter, once a year, once a
decade, . . . 

  

Once per Gradle release would be good.


Adam

[Adam Murdoch-2]

Hans Dockter wrote:

>
> On Sep 18, 2009, at 11:07 AM, Russel Winder wrote:
>
>> On Fri, 2009-09-18 at 11:00 +0200, Hans Dockter wrote:
>>> On Sep 18, 2009, at 10:29 AM, Russel Winder wrote:
>>>
>>>> On Fri, 2009-09-18 at 10:19 +0200, Hans Dockter wrote:
>>>>>>
>>>>>>
>>>>>> - We don't litter .gradle dirs through the source tree. I find this
>>>>>> quite annoying, and I don't imagine I'm alone.
>>>>>
>>>>> Minor point to support this. I really enjoy that GitHub is not
>>>>> polluting every source dir like svn does. I can easily copy stuff
>>>>> around.
>>>>
>>>> This is a Git, Bazaar, Mercurial feature, not a GitHub feature.
>>>
>>> Even I know this ;). Sorry for the typo.
>>
>> No problem, I'm just making sure everyone's awake ;-)
>>
>> Real irritation of the moment is that versions of dependencies of Gradle
>> are now getting scattered around the place, some required but . . .
>> Also many of the version of dependencies used in Gradle are now really
>> very old.
>
> Which ones do you have in mind?
>
>> Should there be a programme of moving to latest versions of
>> everything.  And should this be a continuous activity or something done
>> once a week, once a month, once a quarter, once a year, once a
>> decade, . . .
>
> It should be a continued manual activity rather at the beginning of a
> new release stream.
>
> I have created a Wiki page for this:
> http://docs.codehaus.org/display/GRADLE/Libs+Gradle+Core
>

We could potentially automate this - it would make an interesting
addition to the project reports.


Adam


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email