Gant 0.4 - Creating a powerful DSL for building Java projects

[hdockter]

Hi,

For Gant 0.4 we want to add abstractions as known from Maven, like project layout, build lifecycle, dependencies, etc ...

We want to make Gant 0.4 a powerful build DSL for Java projects, which is a bliss to use. I for myself have grown more and more unhappy with Maven2. Its basic concepts make changing requirements often so hard to implement. Many other things could be said here. We might create an own Wiki page for this. Yet I'm grateful to Maven for being a pioneer in creating a DSL for Java builds. We are going to use many of its concepts. But we think Groovy is a much more suitable base for a build DSL than XML :) 

 

We think this build DSL is also very important for Groovy. Martin Fowler for example is blogging a lot about new ways of doing a build (based on Ruby). The advantages of having a build DSL based on a dynamic language are so enormous, that it won't take long, that more and more people discover this. There are good Ruby projects out there for doing Java builds. This is going to become for many people the first contact with a dynamic language. So Groovy should have something competitive in its stack. 

We are at the very beginning and are thinking about how our DSL should look like and which features should be covered by it.

This is a snippet of a Mail from Russel Winder and my reply:

I wonder if the best way of working is to say:

1.  The goal is to replace the need for Maven and Ant as drivers, but

use all the Ant tasks and the Maven repository.

Agreed. See section on dependency handling above.

2.  We use a few exemplar projects as foci for discussing what the Gant

scripts should look like.

I like this idea very much. The respective projects should be complex enough. Anybody reading this, who is responsible for a build of some open source project and would like to join?  

A friend of mine knows the nutch search engine project very well including its build. He says they have a complex ant based build but does not think there is any interest from the nutch commiters in changing there build. But as a guinea pig, why not. We could at least have a look at it. 

3.  All ideas from SCons, Waf, Rake, Rant, CMake, Maven, Ant, Buildr

that are useful should be considered worth "stealing".

This is what open source is about, isn't it ;)  

Some more of my thoughts on the issue:

Project Layout:

I like the project layout used by Maven2. It is defined here: http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html

One thing that we might add to this, is an additional folder for integration tests (This is also supposed to be added with Maven 2.1).

Lifecycle:

Alternative 1: The Maven way

Maven defines a lifecycle and allows to attach plugins to a lifecycle phase. Depending on the project type (jar, war, ...) there is an immutable set of plugins attached to a lifecycle. Project specific customizations is only possible by adding plugins to the lifecycle. 

 

If we would go for this concept, we should at least make it highly and easily configurable.

Alternative 2: The Rake way

We would have tasks corresponding to lifecycle phases. The tasks are in dependency relations to each other. Each task is executed only once. The Gant dependency mechanism should be sufficient to achieve this goal. Of course Rake goes beyond that.

I think the Maven way is flawed. At least in it current incarnation and at least for one major reason. In Maven there are two ways to execute a task (plugin). Either attached to the lifecycle or calling it directly. If for a direct call a plugin needs prerequisite work, it has to manually call for example the compile lifecycle before it does its work. But if you attach this plugin goal to the lifecycle, the compile lifecycle is executed twice. Therefore you need two goals. One for direct calls, the other when attaching it to the lifecycle. Take for example a look at the assembly plugin.

I favor the Rake way. We should also have a look at buildr (which is based on Rake) how it implements its lifecycle handling.

Dependency handling:

1.) I very much like the way how dependencies are defined in buildr. You define them as part of your script like "LOG4J = log4j:log4j:jar:1.2.14". This allows arbitrary grouping of them in variables, lists, .... By this grouping you can express very good how dependencies are related to each other.

2.) I would like to distinguish between using existing specifications (repository layout) vs. using existing implementations (e.g. Maven Ant tasks). 

a.) I would like to use the Maven2 repository layout and the way Maven2 describes dependencies. A dependency is described with groupid:artifactId:type:version and assumed to be available at a location that maps this to a path as known from Maven2. The remote Maven repo at ibiblio should be the default remote repo to be used.

b.) I would not like to use the Maven Ant Tasks for dependency handling or anything else (see also subsection 3.). If you use it you have to swallow all the Maven handling of dependencies, scopes and snapshots.  

3.) Transitive dependencies

a.) Even if we support transitive dependencies we definitely should offer a mechanism for disabling them. 

b.) If we support transitive dependencies, I don't like the way this is done by Maven2. The policy for resolving dependencies and dealing with version conflicts is carved in stone and is changing with future versions (even with minor releases). The current policies are flawed in my opinion. As the Maven Ant task for retrieving dependencies offers no way to disabled or configure transitive dependency handling, I personally would not use them. The Ivy implementation is much more configurable. We can define our own policy and don't depend one the (volatile) Maven policy. If we use a tool for dependency handling I would go for Ivy. If we don't support transitive dependencies, just for getting/putting dependencies to/from the repository, it might be simple enough not to rely on an external tool for this. The transitive dependency issue deserves a wiki page on its own. But I think the importance of the issue should not be overestimated. 

4.) Snapshots and scope

Maven introduces the concept of snapshots and scopes of dependencies. I would leave them out for the first iteration. If we implement snapshots, I think I prefer the common Ivy way of dealing with this (See http://incubator.apache.org/ivy/doc/bestpractices.html section:Dealing with integration versions). I would not support the scope concept as used by Maven. 

Conclusions

My idea would be to implement a build system with a Maven2 project layout (including multiple subprojects depending on each other) and a a lifecycle task handling based on Gant tasks.  I think having a good DSL for those issues would be the most important thing. On top of that a simple dependency handling for the first release might do. That would be good enough to convince many people to migrate from Ant and Maven. Another question is whether we should start to implement a Groovy Rake as a base for our build system. This would be at least partly in competition to using Ant task (Competition in the sense of different approaches. In a Rake like style the task would not be responsible for checking if a class needs to be compiled. Which is what the Ant task is doing.) But I see no reason why those approaches can't coexist. 

Russel knows such an incredible amount of different build systems. I can't wait on getting his detailed input.

- Hans

[Tom Nichols]

This sounds great.  One comment I have:

I would _very_much_ like to have some sort of transitive dependency
handling.  I use Maven2, and have never run into issues where
transitive handling was a problem.  Not to say problems don't exist --
I've read it a million times so I won't try to deny it.  But when I
use Ant I use the Maven antlibs -- I personally would rather have
Maven's (transitive) dependency handling than none at all.  I started
looking into Ivy but it felt too configuration heavy so I gave up.
Maybe I didn't stick with it long enough.

But that's my take on the matter.
Thanks.
-Tom


On 5/21/07, Hans Dockter <mail@...> wrote:

> Hi,
>
>
> For Gant 0.4 we want to add abstractions as known from Maven, like project
> layout, build lifecycle, dependencies, etc ...
>
> We want to make Gant 0.4 a powerful build DSL for Java projects, which is a
> bliss to use. I for myself have grown more and more unhappy with Maven2. Its
> basic concepts make changing requirements often so hard to implement. Many
> other things could be said here. We might create an own Wiki page for this.
> Yet I'm grateful to Maven for being a pioneer in creating a DSL for Java
> builds. We are going to use many of its concepts. But we think Groovy is a
> much more suitable base for a build DSL than XML :)
>
> We think this build DSL is also very important for Groovy. Martin Fowler for
> example is blogging a lot about new ways of doing a build (based on Ruby).
> The advantages of having a build DSL based on a dynamic language are so
> enormous, that it won't take long, that more and more people discover this.
> There are good Ruby projects out there for doing Java builds. This is going
> to become for many people the first contact with a dynamic language. So
> Groovy should have something competitive in its stack.
>
> We are at the very beginning and are thinking about how our DSL should look
> like and which features should be covered by it.
>
>
> This is a snippet of a Mail from Russel Winder and my reply:
>
>
> I wonder if the best way of working is to say:
>
> 1.  The goal is to replace the need for Maven and Ant as drivers, but
> use all the Ant tasks and the Maven repository.
>
> Agreed. See section on dependency handling above.
>
>
> 2.  We use a few exemplar projects as foci for discussing what the Gant
> scripts should look like.
>
> I like this idea very much. The respective projects should be complex
> enough. Anybody reading this, who is responsible for a build of some open
> source project and would like to join?
>
> A friend of mine knows the nutch search engine project very well including
> its build. He says they have a complex ant based build but does not think
> there is any interest from the nutch commiters in changing there build. But
> as a guinea pig, why not. We could at least have a look at it.
>
>
> 3.  All ideas from SCons, Waf, Rake, Rant, CMake, Maven, Ant, Buildr
> that are useful should be considered worth "stealing".
>
> This is what open source is about, isn't it ;)
>
>
> Some more of my thoughts on the issue:
>
> Project Layout:
>
>
> I like the project layout used by Maven2. It is defined here:
> http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
> One thing that we might add to this, is an additional folder for integration
> tests (This is also supposed to be added with Maven 2.1).
>
> Lifecycle:
>
> Alternative 1: The Maven way
> Maven defines a lifecycle and allows to attach plugins to a lifecycle phase.
> Depending on the project type (jar, war, ...) there is an immutable set of
> plugins attached to a lifecycle. Project specific customizations is only
> possible by adding plugins to the lifecycle.
>
> If we would go for this concept, we should at least make it highly and
> easily configurable.
>
> Alternative 2: The Rake way
> We would have tasks corresponding to lifecycle phases. The tasks are in
> dependency relations to each other. Each task is executed only once. The
> Gant dependency mechanism should be sufficient to achieve this goal. Of
> course Rake goes beyond that.
>
> I think the Maven way is flawed. At least in it current incarnation and at
> least for one major reason. In Maven there are two ways to execute a task
> (plugin). Either attached to the lifecycle or calling it directly. If for a
> direct call a plugin needs prerequisite work, it has to manually call for
> example the compile lifecycle before it does its work. But if you attach
> this plugin goal to the lifecycle, the compile lifecycle is executed twice.
> Therefore you need two goals. One for direct calls, the other when attaching
> it to the lifecycle. Take for example a look at the assembly plugin.
>
> I favor the Rake way. We should also have a look at buildr (which is based
> on Rake) how it implements its lifecycle handling.
>
> Dependency handling:
>
> 1.) I very much like the way how dependencies are defined in buildr. You
> define them as part of your script like "LOG4J = log4j:log4j:jar:1.2.14".
> This allows arbitrary grouping of them in variables, lists, .... By this
> grouping you can express very good how dependencies are related to each
> other.
>
> 2.) I would like to distinguish between using existing specifications
> (repository layout) vs. using existing implementations (e.g. Maven Ant
> tasks).
> a.) I would like to use the Maven2 repository layout and the way Maven2
> describes dependencies. A dependency is described with
> groupid:artifactId:type:version and assumed to be available at a location
> that maps this to a path as known from Maven2. The remote Maven repo at
> ibiblio should be the default remote repo to be used.
> b.) I would not like to use the Maven Ant Tasks for dependency handling or
> anything else (see also subsection 3.). If you use it you have to swallow
> all the Maven handling of dependencies, scopes and snapshots.
>
> 3.) Transitive dependencies
> a.) Even if we support transitive dependencies we definitely should offer a
> mechanism for disabling them.
> b.) If we support transitive dependencies, I don't like the way this is done
> by Maven2. The policy for resolving dependencies and dealing with version
> conflicts is carved in stone and is changing with future versions (even with
> minor releases). The current policies are flawed in my opinion. As the Maven
> Ant task for retrieving dependencies offers no way to disabled or configure
> transitive dependency handling, I personally would not use them. The Ivy
> implementation is much more configurable. We can define our own policy and
> don't depend one the (volatile) Maven policy. If we use a tool for
> dependency handling I would go for Ivy. If we don't support transitive
> dependencies, just for getting/putting dependencies to/from the repository,
> it might be simple enough not to rely on an external tool for this. The
> transitive dependency issue deserves a wiki page on its own. But I think the
> importance of the issue should not be overestimated.
>
> 4.) Snapshots and scope
> Maven introduces the concept of snapshots and scopes of dependencies. I
> would leave them out for the first iteration. If we implement snapshots, I
> think I prefer the common Ivy way of dealing with this (See
> http://incubator.apache.org/ivy/doc/bestpractices.html
> section:Dealing with integration versions). I would not support the scope
> concept as used by Maven.
>
> Conclusions
>
> My idea would be to implement a build system with a Maven2 project layout
> (including multiple subprojects depending on each other) and a a lifecycle
> task handling based on Gant tasks.  I think having a good DSL for those
> issues would be the most important thing. On top of that a simple dependency
> handling for the first release might do. That would be good enough to
> convince many people to migrate from Ant and Maven. Another question is
> whether we should start to implement a Groovy Rake as a base for our build
> system. This would be at least partly in competition to using Ant task
> (Competition in the sense of different approaches. In a Rake like style the
> task would not be responsible for checking if a class needs to be compiled.
> Which is what the Ant task is doing.) But I see no reason why those
> approaches can't coexist.
>
> Russel knows such an incredible amount of different build systems. I can't
> wait on getting his detailed input.
>
> - Hans
>
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

Hi Tom,

I knew that transitive dependencies is going to become a controversial issue :) 

If we implement it, I think transitive dependencies should be expressed via a terminology we offer in our DSL. By that we could for example take the weight from a heavy Ivy configuration :)

But I really don't want to become dependent on the Maven _implementation_. They offer one and only one way of handling dependencies (e.g. how to resolve version conflicts). They are not configurable. The policy how Maven resolves transitive dependencies has changed for example from Maven 2.0.5 to 2.0.6. I could write quite a bit about Maven's dependencies handling. But I'd rather ask Russel to set up a wiki page for this, where we can gather bad existing ideas and good ideas. 

My main concern is, to offer some automagic which sounds good on first sight. But when it comes to complex builds it shows its ugly head. This is what really puts people off with Maven2. And I want to promise those people that they will have a completely different experience when using our tool. So rather no automagic than a flawed automagic. 

We might come up with something that offers a more modest but reliable support for transitive dependency handling. For example a tool that outputs the transitive dependencies on the console and you can copy them if you like into your build script. 

My problem with Ivy is, that I haven't used it. I have just read its documentation. It makes a very good impression to me and seems to offer a better solution for dependency handling than Maven2 (which I have used a lot in complex scenarios). But I would like to have more experience with Ivy.

- Hans

  

On May 21, 2007, at 10:24 PM, Tom Nichols wrote:

This sounds great.  One comment I have:

I would _very_much_ like to have some sort of transitive dependency

handling.  I use Maven2, and have never run into issues where

transitive handling was a problem.  Not to say problems don't exist --

I've read it a million times so I won't try to deny it.  But when I

use Ant I use the Maven antlibs -- I personally would rather have

Maven's (transitive) dependency handling than none at all.  I started

looking into Ivy but it felt too configuration heavy so I gave up.

Maybe I didn't stick with it long enough.

But that's my take on the matter.

Thanks.

-Tom

[Martin Gilday]

As a fairly heavy user of Maven I am all to familiar with where it can
fall short.  Having a tool with some of the concepts behind Maven but
with the power of scripting rather than xml configs could be a wonderful
thing.  

With regards to dependencies making use of existing M2 repositories
would be essential for me in working with and transitioning from
existing projects.  One problem I frequently come across with Maven is
that it relies on everyone else using it for it work well.  By this I
mean if I want to use a framework I need to hope they have put in the
central repo.  Otherwise you need to try and convince them to do it.
The usual suggestion from the Maven mailing list is that you can of
course install to your own local or organisational repo.  But this can
be complicated and time consuming if all you want to do is try something
out.  You have to workout the transtitive dependenices and then install
those as well if they are missing.  What would be great for this would
be the ability to include from a central maven repository but also
simply depend on a few jars you may as with an ant style classpath
include.


----- Original message -----
From: "Hans Dockter" <mail@...>
To: user@...
Date: Mon, 21 May 2007 22:54:21 +0200
Subject: Re: [groovy-user] Gant 0.4 - Creating a powerful DSL for
building Java projects

Hi Tom,

I knew that transitive dependencies is going to become a  
controversial issue :)

If we implement it, I think transitive dependencies should be  
expressed via a terminology we offer in our DSL. By that we could for  
example take the weight from a heavy Ivy configuration :)

But I really don't want to become dependent on the Maven  
_implementation_. They offer one and only one way of handling  
dependencies (e.g. how to resolve version conflicts). They are not  
configurable. The policy how Maven resolves transitive dependencies  
has changed for example from Maven 2.0.5 to 2.0.6. I could write  
quite a bit about Maven's dependencies handling. But I'd rather ask  
Russel to set up a wiki page for this, where we can gather bad  
existing ideas and good ideas.

My main concern is, to offer some automagic which sounds good on  
first sight. But when it comes to complex builds it shows its ugly  
head. This is what really puts people off with Maven2. And I want to  
promise those people that they will have a completely different  
experience when using our tool. So rather no automagic than a flawed  
automagic.

We might come up with something that offers a more modest but  
reliable support for transitive dependency handling. For example a  
tool that outputs the transitive dependencies on the console and you  
can copy them if you like into your build script.

My problem with Ivy is, that I haven't used it. I have just read its  
documentation. It makes a very good impression to me and seems to  
offer a better solution for dependency handling than Maven2 (which I  
have used a lot in complex scenarios). But I would like to have more  
experience with Ivy.

- Hans

On May 21, 2007, at 10:24 PM, Tom Nichols wrote:

> This sounds great.  One comment I have:
>
> I would _very_much_ like to have some sort of transitive dependency
> handling.  I use Maven2, and have never run into issues where
> transitive handling was a problem.  Not to say problems don't exist --
> I've read it a million times so I won't try to deny it.  But when I
> use Ant I use the Maven antlibs -- I personally would rather have
> Maven's (transitive) dependency handling than none at all.  I started
> looking into Ivy but it felt too configuration heavy so I gave up.
> Maybe I didn't stick with it long enough.
>
> But that's my take on the matter.
> Thanks.
> -Tom

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

Hi Martin,

On May 21, 2007, at 11:36 PM, Martin Gilday wrote:

> As a fairly heavy user of Maven I am all to familiar with where it can
> fall short.  Having a tool with some of the concepts behind Maven but
> with the power of scripting rather than xml configs could be a  
> wonderful
> thing.
>
> With regards to dependencies making use of existing M2 repositories
> would be essential for me in working with and transitioning from
> existing projects.

As written in my original mail, we definitely support the Maven  
repository layout. Whether for retrieving single dependencies or in a  
transitive manner is a different question :). At least people should  
be able to disable transitive dependency handling.

> One problem I frequently come across with Maven is
> that it relies on everyone else using it for it work well.  By this I
> mean if I want to use a framework I need to hope they have put in the
> central repo.  Otherwise you need to try and convince them to do it.
> The usual suggestion from the Maven mailing list is that you can of
> course install to your own local or organisational repo.  But this can
> be complicated and time consuming if all you want to do is try  
> something
> out.
> You have to workout the transtitive dependenices and then install
> those as well if they are missing.  What would be great for this would
> be the ability to include from a central maven repository but also
> simply depend on a few jars you may as with an ant style classpath
> include.

The nice thing of our approach is that we don't have to do anything  
difficult to make this possible. We don't have to change any xml  
format :). You can add to your classpath whatever you want in a one  
line statement. We just don't know the syntax yet ;).

- Hans



---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Hans Dockter schrieb:
> Hi,
>
> For Gant 0.4 we want to add abstractions as known from Maven, like
> project layout, build lifecycle, dependencies, etc ...

As  I was encouraging a build tool written in Groovy but never had the
time to really help with Gant I feel a bit guilty ;) Anyway I had of
course some "fantasies" about how such a tool should be and some maven
things don't fit that. One of that is the project layout and plugins. I
very much dislike the maven way of forcing people to make additional
directories for plugins that are used in the build and additional
directories to separate what the developer thought belongs together. I
guess it is no problem to help if the developer keeps a certain layout,
but he should be able to change that very easily.

My basic idea of a lifecycle that time was to have simple method calls:

def build() {
   init()
   compile()
   test()
   jar()
   deploy()
}

where each method can be overwritten and add things to the lifecycle...
Now I know that will not fit, since these lifecycly methods do need
additional informations like descriptions and "task" dependencies. I
guess one diea back then was to have a set of fixed top level methods
for that like

def compile(){
   meta {
     description "call javac task from ant to compiles all java files"
     compilation from:".java", to:".class"
   }
   action {
     ant.javac(...)
   }
}

well, something like that. meta and action would be special methods
activated depending on the phase the build is. Other methods would be
allowed too, marking normal dependencies for the task. Well that was my
idea back then.. of course with some defaults for each method so that
someone could write a class like that:

class CleanAlwaysBuild extends DefaultBuild {
    def build() {
     clean()
     super.build()
   }
}

anyway... it just got the impression that a complete rework of Gant is
on the way, so I thought I mention this again.. just ignore it if you
don't like it ;)

some comments on your points:

[...]

>> 2.  We use a few exemplar projects as foci for discussing what the Gant
>> scripts should look like.
>
> I like this idea very much. The respective projects should be complex
> enough. Anybody reading this, who is responsible for a build of some
> open source project and would like to join?  
>
> A friend of mine knows the nutch search engine project very well
> including its build. He says they have a complex ant based build but
> does not think there is any interest from the nutch commiters in
> changing there build. But as a guinea pig, why not. We could at least
> have a look at it.

My experience with build systems is, that people are interested in it as
long as they need one, and then very fast try to forget about it ;)

>> 3.  All ideas from SCons, Waf, Rake, Rant, CMake, Maven, Ant, Buildr
>> that are useful should be considered worth "stealing".
>
> This is what open source is about, isn't it ;)  

but maybe you guys should point out some things that are good. In any
case you should point out what Gant can do and what not.. I mean there
must be a reason why 0.4 should be so different... or should it be not
so different and I misunderstood something?

> Some more of my thoughts on the issue:
>
> *Project Layout:*
>
> I like the project layout used by Maven2. It is defined
> here: http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
> One thing that we might add to this, is an additional folder for
> integration tests (This is also supposed to be added with Maven 2.1).

yes, that is exactly what I have a problem with... why a java sub
directory? There is more than just java. why a filters directory? I
enver really figured out what that is useful for. having src/main and
src/test is ok... even though I would not name that directory main...
anyway. Just try to make it not too complicated to have mixed content
and a different directory structure, then all should be fine.

[...]
> *Dependency handling:*

use Ivy configured from the build script (no or optional XML)

> 3.) Transitive dependencies

is a must I would say.

> a.) Even if we support transitive dependencies* we definitely should
> offer a mechanism for disabling them.

why?

[...]
 > If we don't support transitive dependencies, just for
> getting/putting dependencies to/from the repository, it might be simple
> enough not to rely on an external tool for this.

I think Ivy is not very big.

> The transitive
> dependency issue deserves a wiki page on its own. But I think the
> importance of the issue should not be overestimated.

well... it will certainly work without that feature too, you will then
have to define all jars instead of the jars you use directly.

> 4.) Snapshots and scope
> Maven introduces the concept of snapshots and scopes of dependencies. I
> would leave them out for the first iteration.

I think the idea of having scopes is a good one.. but I am sure it is
not needed at first. Of course I wouldn't do that completely like maven.
I do not like the idea of having a fixed lifecycle and using the names
and scopes form such a fixed cycle doesn't fit a lifecycle that is not
fixed. For example the idea I described above a scope could be defined
by the name of the method we are in or a list consisting of all
lifecycle method names...  if a list, then you could setup a kind of
inheritance between scopes and if you need only one scope you define it
for the top level method, "build" then.

[...]

> *C**onclusions*
>
> My idea would be to implement a build system with a Maven2 project
> layout (including multiple subprojects depending on each other) and a a
> lifecycle task handling based on Gant tasks.  I think having a good DSL
> for those issues would be the most important thing. On top of that a
> simple dependency handling for the first release might do. That would be
> good enough to convince many people to migrate from Ant and Maven.
> Another question is whether we should start to implement a Groovy Rake
> as a base for our build system.

why depending so much on one tool on one hand and then try to unify to
totally different tools on the other?

bye blackdrag

--
Jochen "blackdrag" Theodorou
Groovy Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Martin Gilday]

I just tried to go to the developer preferences pane and get the
following error.

Plug-in org.codehaus.groovy.eclipse was unable to load class
org.codehaus.groovy.eclipse.core.preferences.DeveloperPage.
Anyone else seeing that?



---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Martin Gilday]

This should have been in response to
Re: [groovy-user] Updated development version of the Groovy Eclipse
plugin available
Sorry

----- Original message -----
From: "Martin Gilday" <martin.lists@...>
To: user@...
Date: Mon, 21 May 2007 23:28:56 +0100
Subject: Re: [groovy-user] Gant 0.4 - Creating a powerful DSL for
building Java   projects

I just tried to go to the developer preferences pane and get the
following error.

Plug-in org.codehaus.groovy.eclipse was unable to load class
org.codehaus.groovy.eclipse.core.preferences.DeveloperPage.
Anyone else seeing that?



---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Michael Baehr-2]

Yes, I just tried and can confirm this.

Martin Gilday schrieb:

> This should have been in response to
> Re: [groovy-user] Updated development version of the Groovy Eclipse
> plugin available
> Sorry
>
> ----- Original message -----
> From: "Martin Gilday" <martin.lists@...>
> To: user@...
> Date: Mon, 21 May 2007 23:28:56 +0100
> Subject: Re: [groovy-user] Gant 0.4 - Creating a powerful DSL for
> building Java   projects
>
> I just tried to go to the developer preferences pane and get the
> following error.
>
> Plug-in org.codehaus.groovy.eclipse was unable to load class
> org.codehaus.groovy.eclipse.core.preferences.DeveloperPage.
> Anyone else seeing that?
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>  

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

Hi Jochen,

On May 22, 2007, at 12:19 AM, Jochen Theodorou wrote:

Hans Dockter schrieb:

Hi,

For Gant 0.4 we want to add abstractions as known from Maven, like project layout, build lifecycle, dependencies, etc ...

As  I was encouraging a build tool written in Groovy but never had the time to really help with Gant I feel a bit guilty ;) Anyway I had of course some "fantasies" about how such a tool should be and some maven things don't fit that. One of that is the project layout and plugins. I very much dislike the maven way of forcing people to make additional directories for plugins that are used in the build and additional directories to separate what the developer thought belongs together. I guess it is no problem to help if the developer keeps a certain layout, but he should be able to change that very easily.

 

See later for directory layout. Re your second point. For me it is clear: No plugins! This artificial separation makes no point for our approach. Often it makes the build less expressive and a paint to maintain. We don't need plugins, we offer good old tasks (targets) in the build script. Groovy offers all that is necessary if one wants to decouple build logic . We don't have to care about that.

My basic idea of a lifecycle that time was to have simple method calls:

def build() {

  init()

  compile()

  test()

  jar()

  deploy()

}

where each method can be overwritten and add things to the lifecycle... Now I know that will not fit, since these lifecycly methods do need additional informations like descriptions and "task" dependencies. I guess one diea back then was to have a set of fixed top level methods for that like

def compile(){

  meta {

    description "call javac task from ant to compiles all java files"

    compilation from:".java", to:".class"

  }

  action {

    ant.javac(...)

  }

}

well, something like that. meta and action would be special methods activated depending on the phase the build is. Other methods would be allowed too, marking normal dependencies for the task. Well that was my idea back then.. of course with some defaults for each method so that someone could write a class like that:

class CleanAlwaysBuild extends DefaultBuild {

   def build() {

    clean()

    super.build()

  }

}

anyway... it just got the impression that a complete rework of Gant is on the way, so I thought I mention this again.. just ignore it if you don't like it ;)

I think I would like to move away from the Maven concept of an external lifecycle object were you attach actions to. I'd rather have lifecycle taks that depend on each other. Those dependencies set up an implicit lifecycle. See my comments on lifecycle in my original mail.

I don't understand the concept of the meta and action closure. Is meta for configuration of the action?

some comments on your points:

[...]

2.  We use a few exemplar projects as foci for discussing what the Gant

scripts should look like.

I like this idea very much. The respective projects should be complex enough. Anybody reading this, who is responsible for a build of some open source project and would like to join?  A friend of mine knows the nutch search engine project very well including its build. He says they have a complex ant based build but does not think there is any interest from the nutch commiters in changing there build. But as a guinea pig, why not. We could at least have a look at it. 

My experience with build systems is, that people are interested in it as long as they need one, and then very fast try to forget about it ;)

3.  All ideas from SCons, Waf, Rake, Rant, CMake, Maven, Ant, Buildr

that are useful should be considered worth "stealing".

This is what open source is about, isn't it ;)  

but maybe you guys should point out some things that are good. In any case you should point out what Gant can do and what not.. I mean there must be a reason why 0.4 should be so different... or should it be not so different and I misunderstood something?

So far Gant is no DSL for building Java projects. There is no terminology for source folder, test folder, lifecycle, packaging, .... We want to introduce this. I think the 'why' and the advantages of such a build DSL are pretty obvious. So Gant 0.4 will hopefully offer many more things than Gant 0.3. This build DSL is the most prominent goal for 0.4. But there is more that can be done. What Rake offers is very appealing to me. Rake offers no concrete tasks like compile. It is completely agnostic whether you use if for Java, C or whatever. It is a base technology for any build or build system. See for example http://www.martinfowler.com/articles/rake.html

I guess Russel can make a lot of points here :)

Some more of my thoughts on the issue:

*Project Layout:*

I like the project layout used by Maven2. It is defined here: http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html

One thing that we might add to this, is an additional folder for integration tests (This is also supposed to be added with Maven 2.1).

yes, that is exactly what I have a problem with... why a java sub directory? There is more than just java. why a filters directory? I enver really figured out what that is useful for. having src/main and src/test is ok... even though I would not name that directory main... anyway.

I was pretty slack to refer to this link. What I mean with using the Maven project layout is assuming the following folder structure:

src/main/java

src/main/resources

src/test/java

src/test/resources

src/integtest/java

src/integtest/resources

src/webapps

That is it. We said that we provide a build DSL for Java projects. Actually I think we should provide a build DSL for Java/Groovy projects. What structure should we offer?

Just try to make it not too complicated to have mixed content and a different directory structure, then all should be fine.

Not too complicated is not good enough. Such simple things should be done simple, otherwise we have failed :)

[...]

*Dependency handling:*

use Ivy configured from the build script (no or optional XML)

3.) Transitive dependencies

is a must I would say.

Why?

a.) Even if we support transitive dependencies* we definitely should offer a mechanism for disabling them. 

why?

[...]

> If we don't support transitive dependencies, just for

getting/putting dependencies to/from the repository, it might be simple enough not to rely on an external tool for this.

I think Ivy is not very big.

The transitive dependency issue deserves a wiki page on its own. But I think the importance of the issue should not be overestimated. 

well... it will certainly work without that feature too, you will then have to define all jars instead of the jars you use directly.

Regarding the whole transitive dependency issue. For me there is only one must. The build script must be able to make it clear what are first level dependencies and what are not. If I wanted to add hibernate to my dependencies in Maven1, I had to declare a list like the following:

hibernate...

ehcache

cglib

asm

....

other non hibernate stuff

If someone decided not to use hibernate any longer, he or she had a good time figuring out which dependencies to remove from the list. How do you know, that asm is not a runtime dependency of another non hibernate dependency. To maintain the dependencies with Maven1 was a nightmare. You had to add a lot of comments. To have more expressiveness in regard to this, was for me initially one of the main advantages of the Maven2 dependency handling. I still think it is better than Maven1, but ...

Finally, some goodies from the Maven dependency handling:

1.) Let's say you want to use dom4j. You want to use the xpath functionality of dom4j. Dom4j needs Jaxen for supporting this. As not everybody uses the xpath functionality, Jaxen is an optional dependency of dom4j.

See: http://repo1.maven.org/maven2/dom4j/dom4j/1.6.1/dom4j-1.6.1.pom

What do you have to do in the pom.xml to get it into your classpath?

<dependency>

   <artifactId>dom4j</artifactId>

...

</dependency>

<dependency>

      <groupId>jaxen</groupId>

      <artifactId>jaxen</artifactId>

      <version>1.1-beta-6</version>

      <optional>true</optional>

 </dependency>

You have to declare Jaxen as a first level dependency and the build expresses no longer, that Jaxen is used in this project only as a dependency of dom4j. Yak!!!! Welcome back in the jar hell!

2.) Let's say you want to use dom4j and some other project. Let's say you want to use an XPP reader with dom4j. As this is an optional dependency, you have to do the same as above. Now imagine the other project has a non optional dependency to xpp3_min. Either you end up with xpp3 and xpp3_min (a subset of xpp3) in your classpath. Or you have to do say:

<dependency>

   <artifactId>dom4j</artifactId>

...

</dependency>

<dependency>

      <groupId>xpp3</groupId>

      <artifactId>xpp3</artifactId>

      <version>1.1.3.3</version>

      <optional>true</optional>

</dependency>

<dependency>

   <artifactId>other project</artifactId>

   <exclusions>

     <exclusion>

         <artifactId>xpp3_min</artifactId>

         <groupId>...<groupId>

      </exclusion>    

  </exclusions>

</dependency>

The latter says on first sight. The other project does not need xpp3. Which is not the case. How to understand the dependencies in a larger build if you have those kind of mechanisms?

This is a common problem. For example cglib and cglib_full, spring and spring-context, spring-....

I feel very strong about not having anything like this in our build system.  For me the Maven dependency mechanism is a no go! Ivy might be an alternative. But I don't know it good enough, to be confident not to run into similar issues. So at least we should be able to disable any transitive dependency handling.!

Now what about this one:

MY_CACHE = "org.cache:mycache:jar:1.0"

HIBERNATE = ["org.hibernate:hibernate:jar:3.2.0.ga",  MY_CACHE, ....]  

OTHER = ["...", MY_CACHE, ....]  

...

compile.with HIBERNATE

This is so simple and expressive.   

(stolen from buildr)

4.) Snapshots and scope

Maven introduces the concept of snapshots and scopes of dependencies. I would leave them out for the first iteration. 

I think the idea of having scopes is a good one.. but I am sure it is not needed at first. Of course I wouldn't do that completely like maven. I do not like the idea of having a fixed lifecycle and using the names and scopes form such a fixed cycle doesn't fit a lifecycle that is not fixed. For example the idea I described above a scope could be defined by the name of the method we are in or a list consisting of all lifecycle method names...  if a list, then you could setup a kind of inheritance between scopes and if you need only one scope you define it for the top level method, "build" then.

[...]

Why not:

JMOCK = group("jmock", "jmock-cglib", :under=>"jmock", :version=>"1.1.0")

JUNIT = "junit:junit:jar:3.8.1"

TEST_DEPS = [JMOCK, JUNIT]

...

compile.with COMPILE_DEPS
test.with TEST_DEPS 

(stolen from buildr ;) )

*C**onclusions*

My idea would be to implement a build system with a Maven2 project layout (including multiple subprojects depending on each other) and a a lifecycle task handling based on Gant tasks.  I think having a good DSL for those issues would be the most important thing. On top of that a simple dependency handling for the first release might do. That would be good enough to convince many people to migrate from Ant and Maven. Another question is whether we should start to implement a Groovy Rake as a base for our build system.

why depending so much on one tool on one hand and then try to unify to totally different tools on the other?

Because Ant and Rake have no intersection. Except that if Rake were available for Java, you could have based Ant on Rake to do things more elegantly. We might want to offer this Rake elegance to our users for there custom tasks. And use it internally for some of our stuff.

I really recommend having a look at buildr.

- Hans

[Alex Shneyderman]

>
> Finally, some goodies from the Maven dependency handling:
>
> 1.) Let's say you want to use dom4j. You want to use the xpath functionality
> of dom4j. Dom4j needs Jaxen for supporting this. As not everybody uses the
> xpath functionality, Jaxen is an optional dependency of dom4j.
>
> See:
> http://repo1.maven.org/maven2/dom4j/dom4j/1.6.1/dom4j-1.6.1.pom
>
> What do you have to do in the pom.xml to get it into your classpath?
> <dependency>
>    <artifactId>dom4j</artifactId>
> ...
> </dependency>
> <dependency>
>       <groupId>jaxen</groupId>
>       <artifactId>jaxen</artifactId>
>       <version>1.1-beta-6</version>
>       <optional>true</optional>
>  </dependency>
>
>
> You have to declare Jaxen as a first level dependency and the build
> expresses no longer, that Jaxen is used in this project only as a dependency
> of dom4j. Yak!!!! Welcome back in the jar hell!

This is exactly why ivy rocks :-) It solves it nicely by allowing you
to decalre configurations of your package and users of your can
depened on one of the configurations you
defined.

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[XMaNIaC]

Some parts of this message have been removed. Learn more about Nabble's security policy.

On 5/22/07, Hans Dockter <mail@...> wrote:

Hi Jochen,

On May 22, 2007, at 12:19 AM, Jochen Theodorou wrote:

So far Gant is no DSL for building Java projects. There is no terminology for source folder, test folder, lifecycle, packaging, .... We want to introduce this. I think the 'why' and the advantages of such a build DSL are pretty obvious. So Gant 0.4 will hopefully offer many more things than Gant 0.3. This build DSL is the most prominent goal for 0.4. But there is more that can be done. What Rake offers is very appealing to me. Rake offers no concrete tasks like compile. It is completely agnostic whether you use if for Java, C or whatever. It is a base technology for any build or build system. See for example  http://www.martinfowler.com/articles/rake.html

I guess Russel can make a lot of points here :)

Just my two cents. I'm normally very kind to abstract/generic architectures but maybe not in this case. In my mind using Groovy to build Java fits like a perfect solution, better than ant, better than Maven. Trying to go beyond that...I don't see it, at all. May be a good solution would be to develop the best build tool for Java. Once we are there we can think about more ambitious goals. I would offer the concrete build tasks (like checkout or compile).

[hdockter]

Some parts of this message have been removed. Learn more about Nabble's security policy.

On May 22, 2007, at 9:47 AM, Jose Noheda wrote:

On 5/22/07, Hans Dockter <mail@...> wrote:

Hi Jochen,

On May 22, 2007, at 12:19 AM, Jochen Theodorou wrote:

So far Gant is no DSL for building Java projects. There is no terminology for source folder, test folder, lifecycle, packaging, .... We want to introduce this. I think the 'why' and the advantages of such a build DSL are pretty obvious. So Gant 0.4 will hopefully offer many more things than Gant 0.3. This build DSL is the most prominent goal for 0.4. But there is more that can be done. What Rake offers is very appealing to me. Rake offers no concrete tasks like compile. It is completely agnostic whether you use if for Java, C or whatever. It is a base technology for any build or build system. See for example  http://www.martinfowler.com/articles/rake.html

I guess Russel can make a lot of points here :)

Just my two cents. I'm normally very kind to abstract/generic architectures but maybe not in this case. In my mind using Groovy to build Java fits like a perfect solution, better than ant, better than Maven. Trying to go beyond that...I don't see it, at all. May be a good solution would be to develop the best build tool for Java. Once we are there we can think about more ambitious goals. I would offer the concrete build tasks (like checkout or compile).

What we want to produce is a good build tool for Java, not for C. No question about this. I have used C as an example to describe that Rake is not language specific. But it has a lot to offer for any kind of build system. It is wasted effort to go into detail her. There are good descriptions put there. If you are interested, start with Martin Fowlers blog about Rake. Buildr for example, a Ruby based build tool for _Java_ is based on Rake. 

- Hans

[Jochen Theodorou]

Hans Dockter schrieb:
[...]
>> def compile(){
>>   meta {
>>     description "call javac task from ant to compiles all java files"
>>     compilation from:".java", to:".class"
>>   }
>>   action {
>>     ant.javac(...)
>>   }
>> }
[...]
> I don't understand the concept of the meta and action closure. Is meta
> for configuration of the action?

the meta is there for basic configuration, yes. For example to make a
simple dry run or for a help page for example...

[...]

> I was pretty slack to refer to this link. What I mean with using the
> Maven project layout is assuming the following folder structure:
>
> src/main/java
> src/main/resources
> src/test/java
> src/test/resources
> src/integtest/java
> src/integtest/resources
> src/webapps
>
> That is it. We said that we provide a build DSL for Java projects.
> Actually I think we should provide a build DSL for Java/Groovy projects.
> What structure should we offer?

I guess I would do something like:

$P/main/src
$P/main/resources
$P/test/src
$P/test/resources
$P/integtest/src
$P/integtest/resources
$P/webapps/src

or alternative

$P/main/java/src
$P/main/groovy/src
...

but I am no big fan of that. And if the time of a combined compiler
comes, then the separation will make less sense.

>> Just try to make it not too complicated to have mixed content and a
>> different directory structure, then all should be fine.
>
> Not too complicated is not good enough. Such simple things should be
> done simple, otherwise we have failed :)

ok ;)

[...]
>>> *Dependency handling:*
>>
>> use Ivy configured from the build script (no or optional XML)
>>
>>> 3.) Transitive dependencies
>>
>> is a must I would say.
>
> Why?

Because it minimizes the configuration you have to do. Dependencies you
  are not using directly tend to be forgotten. Then the lib is updated,
but the dependency is not and such things.

[...]

> Regarding the whole transitive dependency issue. For me there is only
> one must. The build script must be able to make it clear what are first
> level dependencies and what are not. If I wanted to add hibernate to my
> dependencies in Maven1, I had to declare a list like the following:
>
> hibernate...
> ehcache
> cglib
> asm
> ....
> other non hibernate stuff
>
> If someone decided not to use hibernate any longer, he or she had a good
> time figuring out which dependencies to remove from the list.

that's right.

> How do you
> know, that asm is not a runtime dependency of another non hibernate
> dependency. To maintain the dependencies with Maven1 was a nightmare.

or with ant ;)

> You had to add a lot of comments. To have more expressiveness in regard
> to this, was for me initially one of the main advantages of the Maven2
> dependency handling. I still think it is better than Maven1, but ...

m2 is surely better than m1, but that does not mean it is good enough.

> Finally, some goodies from the Maven dependency handling:
>
> 1.) Let's say you want to use dom4j. You want to use the xpath
> functionality of dom4j. Dom4j needs Jaxen for supporting this. As not
> everybody uses the xpath functionality, Jaxen is an optional dependency
> of dom4j.
>
> See:  http://repo1.maven.org/maven2/dom4j/dom4j/1.6.1/dom4j-1.6.1.pom
>
> What do you have to do in the pom.xml to get it into your classpath?
> <dependency>
>    <artifactId>dom4j</artifactId>
> ...
> </dependency>
> <dependency>
>       <groupId>jaxen</groupId>
>       <artifactId>jaxen</artifactId>
>       <version>1.1-beta-6</version>
>       <optional>true</optional>
>  </dependency>
>
>
> You have to declare Jaxen as a first level dependency and the build
> expresses no longer, that Jaxen is used in this project only as a
> dependency of dom4j. Yak!!!! Welcome back in the jar hell!

ah... never thought about such optional jars.. yes, that is right.

> 2.) Let's say you want to use dom4j and some other project. Let's say
> you want to use an XPP reader with dom4j. As this is an optional
> dependency, you have to do the same as above. Now imagine the other
> project has a non optional dependency to xpp3_min. Either you end up
> with xpp3 and xpp3_min (a subset of xpp3) in your classpath. Or you have
> to do say:
>
> <dependency>
>    <artifactId>dom4j</artifactId>
> ...
> </dependency>
> <dependency>
>       <groupId>xpp3</groupId>
>       <artifactId>xpp3</artifactId>
>       <version>1.1.3.3</version>
>       <optional>true</optional>
> </dependency>
>
> <dependency>
>    <artifactId>other project</artifactId>
>    <exclusions>
>      <exclusion>
>          <artifactId>xpp3_min</artifactId>
>          <groupId>...<groupId>
>       </exclusion>    
>   </exclusions>
> </dependency>
>
> The latter says on first sight. The other project does not need xpp3.
> Which is not the case. How to understand the dependencies in a larger
> build if you have those kind of mechanisms?

that should give a conflict.. I mean the build tools should report a
conflict. not because of the exclude, but because xpp3 and xpp3_min both
provide xpp3, but only one provider of xpp3 is allowed at the same
time... well maybe I am thinking too much in the dependency rules used
by debian. In debian you usually don't require a certain version, you
require a package by name and maybe a version range, but usually the
newest version. If now two packages require the same package in
different versions then the newest common version is used. If two
package provide the same "name", then the user selects one of them. But
the debian repository works different from the maven repository. For
debian there is a big file containing all descriptions of all packages
(that is a description of what it is, what name it provides, what
dependecies it has including versions, references to optional parts, the
version of course and then things like author, license and other
things). When doing dependency management, then the tool is downloading
this list and resolving all dependencies before downloading the
packages. Conflicts are resolved and optional parts are selected. In the
maven repository there is no such file, you have to download the poms
one by one and check if there are conflicts and you get no alternative
suggestions, because your tool knows only the requested poms.

[...]

> Now what about this one:
>
> MY_CACHE = "org.cache:mycache:jar:1.0"
> HIBERNATE = ["org.hibernate:hibernate:jar:3.2.0.ga",  MY_CACHE , ....]    
> OTHER = ["...",  MY_CACHE , ....]    
> ...
> compile.with  HIBERNATE
>
> This is so simple and expressive.  
>
> (stolen from buildr)

sure, possible. Looks a bit cryptic, but possible.

bye blackdrag

--
Jochen "blackdrag" Theodorou
Groovy Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[hdockter]

On May 22, 2007, at 12:01 PM, Jochen Theodorou wrote:

> Hans Dockter schrieb:
> [...]
>>> def compile(){
>>>   meta {
>>>     description "call javac task from ant to compiles all java  
>>> files"
>>>     compilation from:".java", to:".class"
>>>   }
>>>   action {
>>>     ant.javac(...)
>>>   }
>>> }
> [...]
>> I don't understand the concept of the meta and action closure. Is  
>> meta for configuration of the action?
>
> the meta is there for basic configuration, yes. For example to make  
> a simple dry run or for a help page for example...
>
> [...]
>> I was pretty slack to refer to this link. What I mean with using  
>> the Maven project layout is assuming the following folder structure:
>> src/main/java
>> src/main/resources
>> src/test/java
>> src/test/resources
>> src/integtest/java
>> src/integtest/resources
>> src/webapps
>> That is it. We said that we provide a build DSL for Java projects.  
>> Actually I think we should provide a build DSL for Java/Groovy  
>> projects. What structure should we offer?
>
> I guess I would do something like:
>
> $P/main/src
> $P/main/resources
> $P/test/src
> $P/test/resources
> $P/integtest/src
> $P/integtest/resources
> $P/webapps/src
>
> or alternative
>
> $P/main/java/src
> $P/main/groovy/src
> ...
>
> but I am no big fan of that. And if the time of a combined compiler  
> comes, then the separation will make less sense.

Good point with the combined compiler. I don't feel strong about  
using java or src.

>
>>> Just try to make it not too complicated to have mixed content and  
>>> a different directory structure, then all should be fine.
>> Not too complicated is not good enough. Such simple things should  
>> be done simple, otherwise we have failed :)
>
> ok ;)
>
> [...]
>>>> *Dependency handling:*
>>>
>>> use Ivy configured from the build script (no or optional XML)
>>>
>>>> 3.) Transitive dependencies
>>>
>>> is a must I would say.
>> Why?
>
> Because it minimizes the configuration you have to do. Dependencies  
> you  are not using directly tend to be forgotten. Then the lib is  
> updated, but the dependency is not and such things.
>
> [...]
>> Regarding the whole transitive dependency issue. For me there is  
>> only one must. The build script must be able to make it clear what  
>> are first level dependencies and what are not. If I wanted to add  
>> hibernate to my dependencies in Maven1, I had to declare a list  
>> like the following:
>> hibernate...
>> ehcache
>> cglib
>> asm
>> ....
>> other non hibernate stuff
>> If someone decided not to use hibernate any longer, he or she had  
>> a good time figuring out which dependencies to remove from the list.
>
> that's right.
>
>> How do you know, that asm is not a runtime dependency of another  
>> non hibernate dependency. To maintain the dependencies with Maven1  
>> was a nightmare.
>
> or with ant ;)
>
>> You had to add a lot of comments. To have more expressiveness in  
>> regard to this, was for me initially one of the main advantages of  
>> the Maven2 dependency handling. I still think it is better than  
>> Maven1, but ...
>
> m2 is surely better than m1, but that does not mean it is good enough.
>
>> Finally, some goodies from the Maven dependency handling:
>> 1.) Let's say you want to use dom4j. You want to use the xpath  
>> functionality of dom4j. Dom4j needs Jaxen for supporting this. As  
>> not everybody uses the xpath functionality, Jaxen is an optional  
>> dependency of dom4j.
>> See:  http://repo1.maven.org/maven2/dom4j/dom4j/1.6.1/dom4j-1.6.1.pom
>> What do you have to do in the pom.xml to get it into your classpath?
>> <dependency>
>>    <artifactId>dom4j</artifactId>
>> ...
>> </dependency>
>> <dependency>
>>       <groupId>jaxen</groupId>
>>       <artifactId>jaxen</artifactId>
>>       <version>1.1-beta-6</version>
>>       <optional>true</optional>
>>  </dependency>
>> You have to declare Jaxen as a first level dependency and the  
>> build expresses no longer, that Jaxen is used in this project only  
>> as a dependency of dom4j. Yak!!!! Welcome back in the jar hell!
>
> ah... never thought about such optional jars.. yes, that is right.
>
>> 2.) Let's say you want to use dom4j and some other project. Let's  
>> say you want to use an XPP reader with dom4j. As this is an  
>> optional dependency, you have to do the same as above. Now imagine  
>> the other project has a non optional dependency to xpp3_min.  
>> Either you end up with xpp3 and xpp3_min (a subset of xpp3) in  
>> your classpath. Or you have to do say:
>> <dependency>
>>    <artifactId>dom4j</artifactId>
>> ...
>> </dependency>
>> <dependency>
>>       <groupId>xpp3</groupId>
>>       <artifactId>xpp3</artifactId>
>>       <version>1.1.3.3</version>
>>       <optional>true</optional>
>> </dependency>
>> <dependency>
>>    <artifactId>other project</artifactId>
>>    <exclusions>
>>      <exclusion>
>>          <artifactId>xpp3_min</artifactId>
>>          <groupId>...<groupId>
>>       </exclusion>      </exclusions>
>> </dependency>
>> The latter says on first sight. The other project does not need  
>> xpp3. Which is not the case. How to understand the dependencies in  
>> a larger build if you have those kind of mechanisms?
>
> that should give a conflict.. I mean the build tools should report  
> a conflict. not because of the exclude, but because xpp3 and  
> xpp3_min both provide xpp3, but only one provider of xpp3 is  
> allowed at the same time... well maybe I am thinking too much in  
> the dependency rules used by debian. In debian you usually don't  
> require a certain version, you require a package by name and maybe  
> a version range, but usually the newest version. If now two  
> packages require the same package in different versions then the  
> newest common version is used. If two package provide the same  
> "name", then the user selects one of them. But the debian  
> repository works different from the maven repository. For debian  
> there is a big file containing all descriptions of all packages  
> (that is a description of what it is, what name it provides, what  
> dependecies it has including versions, references to optional  
> parts, the version of course and then things like author, license  
> and other things). When doing dependency management, then the tool  
> is downloading this list and resolving all dependencies before  
> downloading the packages. Conflicts are resolved and optional parts  
> are selected. In the maven repository there is no such file, you  
> have to download the poms one by one and check if there are  
> conflicts and you get no alternative suggestions, because your tool  
> knows only the requested poms.

In Maven there is no way to express the fact that xpp3 and xpp3_min  
have anything in common. They don't have the same name, so for Maven  
they are as different as groovy.jar from jelly.jar :)

This is a good discussion. Of course I know the appeal transitive  
dependency handling has to people including myself. I would like to  
get a consensus on three issues:

1.) Rather no transitive dependency handling than a bad one.
2.) Not to use the Maven dependency handling (the Maven Ant task).  
I've given some examples why in my former email. There are other  
negative aspects I could point out. But I think the existing examples  
are sufficiently appalling. :)
3.) If there is a good transitive dependency handling tool, let's use  
it. But also enable disabling it.

I gonna have a close look at Ivy which to me seems to do a much  
better job. For one it does distinguish between modules and jars (In  
Maven you have a 1:1 relation of jar:dependencies). With Ivy you can  
have as many modules as you like describing the dependencies of a  
jar. For example there is the hibernate.jar and you have modules like  
hibernate_without_cache, hibernate_with_tangosol, ....

I wonder how Ivy plays together with the Maven based ibiblio  
repository. Does it use it only for the jars? Or does it try to  
evaluate the dependency information in the pom.xml as well?

- Hans



---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Hans Dockter schrieb:
[...]

>>> The latter says on first sight. The other project does not need xpp3.
>>> Which is not the case. How to understand the dependencies in a larger
>>> build if you have those kind of mechanisms?
>>
>> that should give a conflict.. I mean the build tools should report a
>> conflict. not because of the exclude, but because xpp3 and xpp3_min
>> both provide xpp3, but only one provider of xpp3 is allowed at the
>> same time... well maybe I am thinking too much in the dependency rules
>> used by debian. In debian you usually don't require a certain version,
>> you require a package by name and maybe a version range, but usually
>> the newest version. If now two packages require the same package in
>> different versions then the newest common version is used. If two
>> package provide the same "name", then the user selects one of them.
>> But the debian repository works different from the maven repository.
>> For debian there is a big file containing all descriptions of all
>> packages (that is a description of what it is, what name it provides,
>> what dependecies it has including versions, references to optional
>> parts, the version of course and then things like author, license and
>> other things). When doing dependency management, then the tool is
>> downloading this list and resolving all dependencies before
>> downloading the packages. Conflicts are resolved and optional parts
>> are selected. In the maven repository there is no such file, you have
>> to download the poms one by one and check if there are conflicts and
>> you get no alternative suggestions, because your tool knows only the
>> requested poms.
>
> In Maven there is no way to express the fact that xpp3 and xpp3_min have
> anything in common. They don't have the same name, so for Maven they are
> as different as groovy.jar from jelly.jar :)

that's horrible. Didn't they learn from distribution systems used in the
for the Linux variants? Ok, I the thought may have been, that you
compile against one and only one set of fixed jars... but you showed
already that this is not always the case. Of course this means
problems... if we use the maven repository and such an information is
not available... very bad.

> This is a good discussion. Of course I know the appeal transitive
> dependency handling has to people including myself. I would like to get
> a consensus on three issues:
>
> 1.) Rather no transitive dependency handling than a bad one.
> 2.) Not to use the Maven dependency handling (the Maven Ant task). I've
> given some examples why in my former email. There are other negative
> aspects I could point out. But I think the existing examples are
> sufficiently appalling. :)
> 3.) If there is a good transitive dependency handling tool, let's use
> it. But also enable disabling it.

sure

> I gonna have a close look at Ivy which to me seems to do a much better
> job. For one it does distinguish between modules and jars (In Maven you
> have a 1:1 relation of jar:dependencies). With Ivy you can have as many
> modules as you like describing the dependencies of a jar. For example
> there is the hibernate.jar and you have modules like
> hibernate_without_cache, hibernate_with_tangosol, ....

in Debian the hibernate.jar would be a meta package requiring
hibernate_without_cache, hibernate_with_tangosol,...

> I wonder how Ivy plays together with the Maven based ibiblio repository.
> Does it use it only for the jars? Or does it try to evaluate the
> dependency information in the pom.xml as well?

I guess it must resolve that information too.

bye blackdrag

--
Jochen "blackdrag" Theodorou
Groovy Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Tom Nichols]

On 5/22/07, Hans Dockter <mail@...> wrote:
> I wonder how Ivy plays together with the Maven based ibiblio
> repository. Does it use it only for the jars? Or does it try to
> evaluate the dependency information in the pom.xml as well?

IIRC, There is a separate Ivy repo that basically holds the Ivy "POMs"
for artifacts, but the artifacts themselves are still in the Maven
repo.
http://www.jaya.free.fr/ivyrep/
But it seems to me that there is a lot less there than what is in
Maven.  I believe Ivy will read Maven POMs in the absence of Ivy
files.  But in that case it would seem Ivy is limited to the same
information that Maven is.

Not sure if this has already been asked but -- Maven comes with
_a_lot_ of good stuff.  Is it so fundamentally broken that it would
not be worth fixing?  I mean, help fix Maven and build on top of that,
or start from scratch while still trying to use the Maven repo and
POMs for dependency handling...  It sounds like a lot of people think
Maven is a lot cause at this point :(

-Tom

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Tom Nichols schrieb:
> On 5/22/07, Hans Dockter <mail@...> wrote:
[...]
> Not sure if this has already been asked but -- Maven comes with
> _a_lot_ of good stuff.  Is it so fundamentally broken that it would
> not be worth fixing?

maybe it would be worth it, but when you say the plugin concept itself
is the wrong way, then how does it fit maven? That is something so
fundamental... I mean of course you can say this and that is a bug, even
provide patches and so, but not everything you consider to be a bug is a
bug in the view of the developers. If the ideology behind it is
incompatible, then it is not worth fixing.

> I mean, help fix Maven and build on top of that,
> or start from scratch while still trying to use the Maven repo and
> POMs for dependency handling...  It sounds like a lot of people think
> Maven is a lot cause at this point :(

I guess we wouldn't use the maven repo if it were not such a good source
of jars.

bye blackdrag

--
Jochen "blackdrag" Theodorou
Groovy Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Tom Nichols]

On 5/22/07, Jochen Theodorou <blackdrag@...> wrote:
> > I mean, help fix Maven and build on top of that,
> > or start from scratch while still trying to use the Maven repo and
> > POMs for dependency handling...  It sounds like a lot of people think
> > Maven is a lot cause at this point :(
>
> I guess we wouldn't use the maven repo if it were not such a good source
> of jars.

This is the problem -- I think you _have_ to use the maven repo for
the JARs -- good luck getting that many projects published in a second
repository.  So we pretty much have to accept that we are limited to
POMs and/or Ivy files where available if we want to do any sort of
dependency handling I think.

That is, unless the Gant DSL is _sooo_ good that project maintainers
just _want_ to write their dependencies in Gant!  Hey, I suppose it
could happen :P

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Tom Nichols]

On 5/22/07, Hans Dockter <mail@...> wrote:
> Finally, some goodies from the Maven dependency handling:
>
> 1.) Let's say you want to use dom4j. You want to use the xpath functionality
> of dom4j. Dom4j needs Jaxen for supporting this. As not everybody uses the
> xpath functionality, Jaxen is an optional dependency of dom4j.
....
> You have to declare Jaxen as a first level dependency and the build
> expresses no longer, that Jaxen is used in this project only as a dependency
> of dom4j. Yak!!!! Welcome back in the jar hell!
>
> 2.) Let's say you want to use dom4j and some other project. Let's say you
> want to use an XPP reader with dom4j. As this is an optional dependency, you
> have to do the same as above. Now imagine the other project has a non
> optional dependency to xpp3_min. Either you end up with xpp3 and xpp3_min (a
> subset of xpp3) in your classpath. Or you have to do say:
...

So for optionals you need to specify a direct dependency, and for
modules with different names, you have to add an exclude for one of
them.  Yes, I can see how this is a pain (especially if several
dependencies used xpp_min), but I don't feel it's a show-stopper.  I
think I'd still rather add some excludes than have no transitive
dependency handling.

How does buildr deal with this?  It still must use the Maven POMs.

Maven has some other things that I don't necessarily like, such as
limited scopes and the inability to have multiple top-level source
folders.  But Maven's dependency handling isn't all that bad.

How about this --
Would it be reasonable to make this functionality pluggable?  i.e.
Allow Maven to handle it, or allow Ivy or some other strategy?

To be honest, I'm not very familiar with Maven's internals _at_all_
and I don't think I could contribute much more than this discussion to
the development effort.  So I understand how much my words are worth.
But I just hate to see things re-invented unless they have to be.

Thanks.
-Tom

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email