Re: Performance of Gradle vs. Maven

> On Thu, 2009-06-11 at 08:54 +0200, Hans Dockter wrote:
> [ . . . ]
>>> gradle clean libs ~ 12s
>>> mvn clean package ~ 8s
>>>
>>> gradle libs ~ 10 - 20s (This one varied quite a bit while testing)
>>> mvn package ~ 5s
>
>
> The following is not an experiment as such just a summary of a few  
> data
> points (so more than just one run taken at random).
>
> For my ADS library (which is pure Java with TestNG) on my 3GHz P4 with
> NFS mounted filestore, I find:
>
> "gradle clean" of a fully built project ~ 4--6s
> "mvn clean" of a fully built project ~ 4--5s
>
> "gradle clean" of an already clean project ~ 4--4.5s
> "mvn clean" of an already clean project ~ 1--2s
>
> "gradle -i libs" from a cleaned stated ~ 15--23s
> "mvn package" from a cleaned state ~ 13--15s
>
> "gradle -i libs" of an already built project ~ 8.5--10.5s
> "mvn package" of an already built project ~ 6--7s
>
> This is the time reported by the application and is clearly only
> partially related to the time it actually takes from issuing the  
> command
> to getting the next prompt.  A full experiment would measure reported
> time and actual operating system recorded time.
>
> So overall the impression that Gradle is a bit slower than Maven is
> backed up, at least for this project in its current state.
>
> I have long ago deleted the Ant and Gant builds so I cannot compare
> those.
>
> [ . . . ]
>
>> The performance of a build tool has a couple of aspects. One is its
>> execution efficiency. The other is its smartness in not doing
>> unnecessary actions.
>
> True but in what situations is this an issue?  Given the indicators
> above, Gradle is still only just keeping up with Maven for a
> straightforward, pure Java project of relatively small size.

>
>> I don't think there are major usability issues with Gradle's  
>> execution
>> efficiency. On the other hand we haven't focused on optimizing that  
>> in
>> the last releases. I was doing some profiling recently and found a
>> significant inefficiency in some areas were we use reflection. The  
>> fix
>> is now in trunk and has a very nice impact on the performance.
>
> Gradle's performance is definitely in the "not bad" category, it is
> perfectly usable.  However, it does appear to be slower than Maven for
> the cases I use it for, which is sad.
>
>> I use the Apache Abdera project (which is a 28 projects multi-project
>> build) as my guinea pig project for comparisons.
>>
>> gradle clean libs -Dskip.test takes 15 seconds on my machine. (Gradle
>> trunk)
>> mvn clean package -o takes 20 seconds. (Maven 2.1)
>>
>> gradle libs -Dskip.test takes 7 seconds on my machine. (Gradle trunk)
>> mvn package -o takes 9 seconds. (Maven 2.1)
>
> I don't know whether the -Dskip.test actually makes a difference to  
> the
> Gradle run or the -o to the Maven run, but it is clear that the Gradle
> run is not directly analogous to the Maven run so this data point  
> could
> be undermined.

>
>> So this is looking very good I think. There are more optimizations
>> that can be done to improve execution efficiency. We definitely need
>> to figure out why the testing takes so much longer than with Ant. Its
>> funny that Maven has the same problem.
>>
>> Another area were we want to improve Gradle in not doing unnecessary
>> stuff. Having a good change detection and act on this accordingly,
>> etc ....
>
> I must reiterate that I don't think Gradle has a performance problem  
> per
> se, but I do think it would be better if we can get Gradle performing
> faster than Maven in real use projects and workflows -- as opposed to
> benchmarking.  It would be good to be faster in benchmarking as well  
> of
> course :-)
>
>>> (2) maybe Gradle just has a slow startup time and with bigger
>>> projects it's not an issue, (3) maybe I'm doing/not doing something
>>> that is making the build slower.
>>>
>>> I obviously didn't do any detailed test, but I did notice that in my
>>> small build the steps that seem to take the most time are the
>>> initial startup, the test task, and the jar task.
>
> I suspect there is a Groovy startup cost.  I have no data points to  
> back
> up this opinion, but at every turn Groovy execution tends to show a
> startup cost over Java execution.
>
>>> How do I tell Gradle to fork once for the tests instead of the
>>> default per test forking?
>>
>> test {
>>     options.fork(forkMode: ForkMode.ONCE)
>> }
>>
>> or
>>
>> test.options.fork(forkMode: ForkMode.ONCE)
>
> Is this true for TestNG tests or just JUnit tests?
>
> I suspect not since my project adding this line leads to:
>
> Cause: No signature of method: groovy.xml.MarkupBuilder.fork() is
> applicable for argument types: (java.util.LinkedHashMap) values:
> {["forkMode":once]}