« Return to Thread: Task Optimization

Re: Task Optimization

by Steve Appling :: Rate this Message:

Reply to Author | View in Thread

I have a proof of concept implementation of some of this at
git://github.com/sappling/gradle.git in the "opt" branch.

This includes:
1) A new onlyIf method on Task
2) A new didWork method on Task
3) Implementations of didWork for Compile and GroovyCompile.  I don't think that
we can handle Ant's Copy task in this same way.  We may have to use a
replacement, but this has other consequences.
4) Changes to src/samples/java/quickstart/build.gradle to demo didWork and onlyIf.
5) A start at OptimizationHelper.isNeeded method.  This will require some
additional dependency management features, so I stopped development until I got
some feedback on this whole approach.

Steve Appling wrote:

> I am interested in ways to short circuit task execution for the purpose
> of optimization.  I would love to see some of this in 0.7 and would be
> glad to contribute.
>
> Here are some ideas:
> 1) Add an "onlyIf" method to Task that is given a closure. The closure
> would be executed before the first action of the task and would cancel
> execution of the task (with appropriate lifecycle message) if it
> returned false.  This closure would have as a delegate an optimization
> container with some helper methods that would provide more convenient
> access to change detection (among other things). Then you could do:
>   mytask.onlyIf {
>     timestampChanged 'src/main/mysrc'
>     // or contentsChanged 'src/main/mysrc'
>   }
>
> 2) Running a clean should probably remove the change detection state
> information for a project (or at least the clean task should be able to
> be configured to do this conveniently).
>
> 3) I would like some general way for tasks to indicate that they did
> anything. Perhaps task.getDidWork().  BTW, I figured out how to do this
> for gradle's use of ant.javac and can now tell if it really compiled
> anything.
>
> 4) I would like to be able to specify that a chain of dependent tasks
> only execute a task if Task.didWork is true for all of its dependents.  
> Note that this is not always desired, so you need to be able to turn
> this on and off.  I'm not sure of the best way to configure this.  If we
> use the onlyIf method suggested above, it might take another closure to
> check this that would be returned from a  "needed" method.  This would
> look like:
>   myTask.onlyIf(needed())
>
> This probably should be the default for tests, but perhaps not for all
> Tasks.
>
> Javac is already checking to see if the source files are out of date
> with the classes, so I don't think that the javac task needs to use the
> new changedetection.  This would, however let you stop other tasks in
> the chain (like test) if nothing needed to be compiled.  (unrelated: I
> would also like to see an option on compile to use Ant's depend task.  I
> think the current dependencyTracking option doesn't work with the modern
> compiler. )
>
> Other types of tasks could make good use of Tom's change detection.
>
> 5) We probably want a command line option to be able to disable all of
> these optimizations.  Sometimes you really want to force a build with no
> optimizations (without running clean).
>
>
> In the race for speed, Gradle will probably never catch Ant in a clean
> build (at least while you are delegating most of the expensive stuff to
> ant).  However, most of the time developers are doing incremental
> changes on existing systems and not running clean.  In this case, if
> Gradle can support features to conveniently bypass unneeded steps, it
> can be much faster.  Also, Gradle has a huge advantage of a more
> maintainable and modular build specification.
>

--
Steve Appling
Automated Logic Research Team

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


 « Return to Thread: Task Optimization