RE: [RANT] This Maven thing is killing us....

[Niclas Hedhman]

On Tuesday 04 July 2006 20:53, Carlos Sanchez wrote:
> If project A says it depends on B 1.0 and C says it depends on B 1.1,
> there's a conflict in Maven, Ant and anything you want to use, the
> difference is that Maven tries to do it for you, but you still can
> override that behaviour.

Well, since Cocoon is going for OSGi, this does not need to be a problem per
se. However, Maven currently doesn't support the CP resolution needed in
complex OSGi builds (such as Eclipse itself). There are people investigating
what would be needed, but no promises have been made so far.

One main issue is that OSGi is not concerned over versioning of the jars, but
versioning of the packages (normal Java packages for the uninitiated). More
exactly, the jar is essentially just a delivery container of the packages,
and the resulting "classpath" may be a subset of a jar and mixed in with
packages from another version of the same jar.

For the record; Ant is not capable of handling this perfectly either.
Also; Very often it is not need for OSGi development.

Cheers
Niclas

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Jason van Zyl-2]

On 4 Jul 06, at 1:45 PM 4 Jul 06, Steve Loughran wrote:
>
> In a way, many of the stuff in M2 is experimental; a build tool  
> that effectively encodes beliefs about how a project should be  
> structured and delivered, focusing on component-based development  
> instead of application dev. I also think its time to look at how  
> well some of the experiment is working.
>

You make it sound like we're some sort of cult :-)

The phrase "encoding beliefs" is an inaccurate description. It's is  
simply the pursuit of best practices for software development and  
those practices are very much mutable, this thread being very good  
evidence of that. We also not only focused on component-oriented  
development, we ourselves develop applications ourselves and we're  
trying to make that coherent as well.

> Personally, I always experience a bit of fear when adding a new  
> dependency to a project. the repository stuff, and estimate a  
> couple of hours to get every addition stable, primarily by building  
> up a good exclusion list.

This is the place to talk about that as people shouldn't be fearful  
adding dependencies. But people who have an ideal setup here they  
completely control the repository they use internally don't have many  
of the problems that people are experiencing in this thread. Having a  
public repository of high quality is not a trivial task.

>
> Is it worse than before? Better? Or just, well, different? and if  
> things are either worse or not as good as they could be, what can  
> be changed?
>

The process is absolutely better. The process couple with the public  
infrastructure we have now is problematic. Two very different things.

> One underlying cause seems to be pom quality. Open source software  
> dev is a vast collection of loosely coupled projects, and what we  
> get in the repository in terms of metadata matches this model. Each  
> project produces artifacts that match their immediate needs, with  
> POM files that appear to work at the time of publishing. Maven then  
> caches those and freezes that metadata forever, even if it turns  
> out that the metadata was wrong.  There's far better coherence  
> within Gump, where the metadata is effectively maintained more by  
> the gump team themselves than by the individual projects.

There is absolutely no way this is scalable over time. You are saying  
that a small group of people can maintain metadata for projects that  
they are not intimately involved with? That's like saying that people  
who live outside your community have a better chance at describing  
your community. I really just don't think that's possible. How many  
problems has Gump had over the years trying to maintain the metadata?  
Huge problems, almost never in sync with projects. You basically find  
out when it breaks and go back track most of the time. There is no  
doubt that the same process will happen with Maven where users of  
Maven will eventually make their metadata better but that will take  
time. Gump has been around for 5-6 years now. People are really only  
starting to use Maven 2.x which is closing in on being out for a  
year. I am will to bet in another year a great number of the problems  
seen in this thread will be gone. I would argue that Gump will not  
work precisely because it is not the projects themselves maintaining  
the metadata. Projects using Maven will eventually get it right  
because it provides some value to them to get it right.

>
> Question is, what to do about it? And if the m2 repository was an  
> attempt to leave the problems of the M1 repository behind, has it  
> worked?
>

To a large extent I would say we have fixed many of the problems on a  
technical level. Correctly the metadata and educating projects as to  
how best maintain is it is a social problem and a matter of  
education. Couple that with some automated integrity checks that will  
be performed by the repository manager.

> -steve
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>

Jason van Zyl
jason@...




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Jason van Zyl-2]

On 4 Jul 06, at 2:37 PM 4 Jul 06, Steve Loughran wrote:

> Carlos Sanchez wrote:
>> The repository is as good as the users/projects make it. There's no
>> difference at all with using ant and including the wrong jars, maybe
>> the problem is that how to fix it in maven is not as easy as in ant.
>> If project A says it depends on B 1.0 and C says it depends on B 1.1,
>> there's a conflict in Maven, Ant and anything you want to use, the
>> difference is that Maven tries to do it for you, but you still can
>> override that behaviour.
>
>
> It seems to me that the difference in ant, the duty to set up your  
> classpath belongs to the project alone, so you, the build.xml  
> author are the only one who can make a mess of your CP.

This also means that you are the only one who can help create a  
correct classpath.

>
> However, on any system with transitive dependencies, you are ceding  
> control of your classpath to other programs out there.

No, you are ceding control of the dependency chain to the projects  
which you are depending on.

> Even if you think you know exactly what the dependency graph of  
> your app is, an update to a new version of any your dependencies  
> can pull in new metadata, with a new set of dependencies, and  
> potentially a new classpath.

So how is that different then with Ant when you try a new version of  
a dependency? You're going to have to go hunt down the dependencies  
or copy in a new set of dependencies provided by that project. It  
doesn't matter what you use to build if you change a dependency you  
are subject to change.

>
> This is not a maven-specific problem; anything that supports  
> transitive dependency logic can suffer from it. Gump and Ivy could,  
> though in both cases the descriptors tend to be hand-written tuned  
> to not exhibit the problem. (in gump most projects dont export  
> dependencies, as the default is compile-time-only).
>

Ok, I agree this is not a Maven specific problem.

You cannot maintain metadata for a growing number of projects and you  
can't stay on top of projects changing all the time except in  
reaction of problems. The projects themselves must provide useful  
metadata and it is of value to them to do so.

> > Right now we are in a good position with a huge number of users  
> trying
> > and testing the metadata in the repository, and forcing projects to
> > support maven by providing good data.
>
> That still assumes that transitive dependencies are a good thing,  
> and that perfect metadata is achievable. I'm not sure about either  
> of those. I also think they're straying dangerously close to one of  
> the big software engineering tar-pits: versioning.
>

Transitive dependencies in the state they are in now can be  
problematic. What can be done to fix this? The repository manager  
running integrity checks, some visualization tools to help people see  
the chains easily, and visualization tools to help detect conflicts.  
For both of these things IDE integration will be key to making this  
easy for users. Big challenges, and not trivial but certainly not  
untenable. The metadata will never be perfect but right now I still  
think it's far from being ideal because we have no real active  
process of improving it on a large scale. Carlos puts in a _lot_ of  
time trying to correct things and absorb changes submitted for  
improvement but as mentioned in the previous message it's a matter of  
education and automated tools running to point people in the right  
direction.

> -steve
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>

Jason van Zyl
jason@...




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Ralph Goers]

Carlos Sanchez wrote:
> If project A says it depends on B 1.0 and C says it depends on B 1.1,
> there's a conflict in Maven, Ant and anything you want to use, the
> difference is that Maven tries to do it for you, but you still can
> override that behaviour.
Actually, you can't in Maven 2 - at least not in any rational manner -
until http://jira.codehaus.org/browse/MNG-1577 is addressed.  The good
news is that there is a strong desire to move to Maven 2 in my
organization, so I may be given some time soon to do something about this.

However, this isn't even the biggest problem that has been hampering the
Cocoon community. It is that there seems to be at best a 50% chance of
getting a Maven 2 based Cocoon build to work due to dependencies failing
to download from repositories.  Cocoon also consistently uses snapshot
versions of some of its dependencies. While this isn't desirable, it is
a fact of life.  From Cocoon's perspective we really need to treat them
like they are a "private" release - i.e. they weren't formally released
from wherever they came from, but we don't want to have to download them
at every build since they will never change.

Ralph

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[jrduncans]

On 7/4/06, Jason van Zyl <jason@...> wrote:

>
> On 4 Jul 06, at 2:37 PM 4 Jul 06, Steve Loughran wrote:
>
> The metadata will never be perfect but right now I still
> think it's far from being ideal because we have no real active
> process of improving it on a large scale. Carlos puts in a _lot_ of
> time trying to correct things and absorb changes submitted for
> improvement but as mentioned in the previous message it's a matter of
> education and automated tools running to point people in the right
> direction.

Well, but it seems (recently?) that a policy has been put into place
that POMs already in the repository should not be corrected or
improved, in order to preserve repeatability for builds depending on
the existing version, and that corrections should be done by making
new releases.  It's hard enough to get projects to care about
providing Maven POMs, but to ask for a new release seems a bit much.

It also may seem ideal to have projects take care of their own POMs,
but it makes it frustrating for users to provide information on fixes.
 I know, personally, I've cut down on contributing to central
repository improvement.  I've taken to simply installing new jars to
my internal repository, because asking individual projects to do it
gives slow-to-no returns.  I put top-level exclusions into
dependencyManagement rather than request changes to POMs, because
again, there seems no process for actually getting that to happen
that's not haphazard.  I'll try to work on doing better, but the
cost-reward ratio isn't helping.

I think maybe some either feature or convention for handling version
changes to just POMs so they can be improved without another release
of the software would help.  Some clarification/policy statements on
when I should go straight to the project responsible for a jar vs.
filing in Maven evangelism for uploads & for POM improvements might be
helpful.  Certainly some of the 2.1 planned features (like being able
to rely on geronimo-transaction & have that take care of anything
relying in javax.transaction:jta...) could help.  I think some concept
work needs to go into optional dependencies, because it we can't
control when Spring decides they want to stop providing modularized
jars, and move to a single jar that will essentially have all optional
dependencies.  I'm not looking forward to getting my projects to work
with Spring 2.0.

Believe me, this is all coming from someone who's been trying.  I've
filed bug reports with Spring, and Lucene, other projects to get Maven
uploads.  I've volunteered to work on providing and maintaining a
Maven 2 build for an incubator project so that it will be easy to
provide Maven jars & poms when the time comes.

Oh, another quality issue.  -source and -javadoc jars.  It really
slows down running eclipse:eclipse when half or more of my
dependencies don't have these jars.  And a lot don't.  All of
spring-1.2.7, for instance.

My purpose isn't just to complain.  I just think that there's going to
have be more to it than "it'll get better over time" for the central
repository to improve, because, from the narrow view of the things I
use, it's getting a bit worse, not better right now.

--
Stephen Duncan Jr
www.stephenduncanjr.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Carlos Sanchez-4]

On 7/5/06, Stephen Duncan <stephen.duncan@...> wrote:

> On 7/4/06, Jason van Zyl <jason@...> wrote:
> >
> > On 4 Jul 06, at 2:37 PM 4 Jul 06, Steve Loughran wrote:
> >
> > The metadata will never be perfect but right now I still
> > think it's far from being ideal because we have no real active
> > process of improving it on a large scale. Carlos puts in a _lot_ of
> > time trying to correct things and absorb changes submitted for
> > improvement but as mentioned in the previous message it's a matter of
> > education and automated tools running to point people in the right
> > direction.
>
> Well, but it seems (recently?) that a policy has been put into place
> that POMs already in the repository should not be corrected or
> improved, in order to preserve repeatability for builds depending on
> the existing version, and that corrections should be done by making
> new releases.  It's hard enough to get projects to care about
> providing Maven POMs, but to ask for a new release seems a bit much.

There's always the possibiblity of adding a new version ourselves with
a build number, like 1.0-1 where only the pom changes. Only as an
exception when a project is dead or don't release often

>
> It also may seem ideal to have projects take care of their own POMs,
> but it makes it frustrating for users to provide information on fixes.
>  I know, personally, I've cut down on contributing to central
> repository improvement.  I've taken to simply installing new jars to
> my internal repository, because asking individual projects to do it
> gives slow-to-no returns.  I put top-level exclusions into
> dependencyManagement rather than request changes to POMs, because
> again, there seems no process for actually getting that to happen
> that's not haphazard.  I'll try to work on doing better, but the
> cost-reward ratio isn't helping.

Well, if everybody does that then there's no community benefit and we
all lose. When I need something not in ibiblio I put it there because
it's not much harder than putting it in my local repo.

>
> I think maybe some either feature or convention for handling version
> changes to just POMs so they can be improved without another release
> of the software would help.  Some clarification/policy statements on
> when I should go straight to the project responsible for a jar vs.
> filing in Maven evangelism for uploads & for POM improvements might be
> helpful.  Certainly some of the 2.1 planned features (like being able
> to rely on geronimo-transaction & have that take care of anything
> relying in javax.transaction:jta...) could help.  I think some concept
> work needs to go into optional dependencies, because it we can't
> control when Spring decides they want to stop providing modularized
> jars, and move to a single jar that will essentially have all optional
> dependencies.  I'm not looking forward to getting my projects to work
> with Spring 2.0.

You can always have poms for different maven configurations against
only one jar, see
http://jroller.com/page/carlossg?entry=optional_dependencies_in_maven

>
> Believe me, this is all coming from someone who's been trying.  I've
> filed bug reports with Spring, and Lucene, other projects to get Maven
> uploads.  I've volunteered to work on providing and maintaining a
> Maven 2 build for an incubator project so that it will be easy to
> provide Maven jars & poms when the time comes.

You can see in http://opensource.atlassian.com/projects/spring/browse/SPR-1484
that Spring got pressure from their users to provide POMs, it's by far
the most popular issue. Any project that wants to succeed must listen
to their users

>
> Oh, another quality issue.  -source and -javadoc jars.  It really
> slows down running eclipse:eclipse when half or more of my
> dependencies don't have these jars.  And a lot don't.  All of
> spring-1.2.7, for instance.

You can download Spring, jar the stuff up and submit them for adding
to ibiblio. I don't understand how can you complain when maven is the
only project providing that feature.

>
> My purpose isn't just to complain.  I just think that there's going to
> have be more to it than "it'll get better over time" for the central
> repository to improve, because, from the narrow view of the things I
> use, it's getting a bit worse, not better right now.

My view is that it's getting better, more jars, more poms, more
javadocs, more sources, and more and more projects caring about the
repository, being a must for them.

>
> --
> Stephen Duncan Jr
> www.stephenduncanjr.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>

--
I could give you my word as a Spaniard.
No good. I've known too many Spaniards.
                             -- The Princess Bride

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Carlos Sanchez-4]

On 7/4/06, Ralph Goers <Ralph.Goers@...> wrote:

>
>
> Carlos Sanchez wrote:
> > If project A says it depends on B 1.0 and C says it depends on B 1.1,
> > there's a conflict in Maven, Ant and anything you want to use, the
> > difference is that Maven tries to do it for you, but you still can
> > override that behaviour.
> Actually, you can't in Maven 2 - at least not in any rational manner -
> until http://jira.codehaus.org/browse/MNG-1577 is addressed.  The good
> news is that there is a strong desire to move to Maven 2 in my
> organization, so I may be given some time soon to do something about this.

Yes you can, it's not the best way to do it but you can, by adding
explicitly the dependency with the versoin you want to your pom. In
the very worst case you have to add all transitive deendencies to your
pom, like in Maven 1.

>
> However, this isn't even the biggest problem that has been hampering the
> Cocoon community. It is that there seems to be at best a 50% chance of
> getting a Maven 2 based Cocoon build to work due to dependencies failing
> to download from repositories.  Cocoon also consistently uses snapshot
> versions of some of its dependencies. While this isn't desirable, it is
> a fact of life.  From Cocoon's perspective we really need to treat them
> like they are a "private" release - i.e. they weren't formally released
> from wherever they came from, but we don't want to have to download them
> at every build since they will never change.

You can publish that releases under your groupId, anything you want,
you can't force things to happen under other groupIds. If the other
projects don't want to make a release it doesn't matter if you use
maven or anything else.

>
> Ralph
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>


--
I could give you my word as a Spaniard.
No good. I've known too many Spaniards.
                             -- The Princess Bride

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[brettporter]

Ralph,

Thanks for this, it's very helpful.

On 5/07/2006 6:59 AM, Ralph Goers wrote:
> However, this isn't even the biggest problem that has been hampering the
> Cocoon community. It is that there seems to be at best a 50% chance of
> getting a Maven 2 based Cocoon build to work due to dependencies failing
> to download from repositories.  Cocoon also consistently uses snapshot
> versions of some of its dependencies. While this isn't desirable, it is
> a fact of life.  From Cocoon's perspective we really need to treat them
> like they are a "private" release - i.e. they weren't formally released
> from wherever they came from, but we don't want to have to download them
> at every build since they will never change.

The first thing I'd suggest is for those having problems to try another
mirror. I know requiring everyone to do that is a pain and not a long
term solution, but I'd like to see how much that reduces the problem.

Regarding private releases, this is entirely possible just by giving
them a version that isn't in the form of a snapshot. eg. if it is jetty
6.0-SNAPSHOT, use 6.0-cocoon-1, 6.0-cocoon-2.

We'd considered this on the asf repository list:
http://mail-archives.apache.org/mod_mbox/www-repository/200606.mbox/%3c31cc37360606062345n4a1dcd8eq1a5277590b858c33@...%3e

However, this changes the use case a bit, since it considered those only
to be needed for development and full releases would be released against
full releases of their dependencies.

It depends on how you use them as to the best solution here. I assume
that they are customised for cocoon, so they shouldn't be considered to
be the same as the original. In that case, I'd suggest you release them
under your own groupID (maybe org.apache.cocoon.thirdparty) so that they
don't conflict (bearing in mind that someone will transitively receive
both that and the original if they are both used). I'm assuming this
isn't the case as you are only bundling these modified versions into a
distributable app, not as part of a reusable component?

Hope this helps,
Brett

--
Brett Porter <brett@...>
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[djencks]

On Jul 4, 2006, at 5:16 PM, Carlos Sanchez wrote:

> On 7/5/06, Stephen Duncan <stephen.duncan@...> wrote:
>> On 7/4/06, Jason van Zyl <jason@...> wrote:
>> >
>> > On 4 Jul 06, at 2:37 PM 4 Jul 06, Steve Loughran wrote:
>> >
>> > The metadata will never be perfect but right now I still
>> > think it's far from being ideal because we have no real active
>> > process of improving it on a large scale. Carlos puts in a _lot_ of
>> > time trying to correct things and absorb changes submitted for
>> > improvement but as mentioned in the previous message it's a  
>> matter of
>> > education and automated tools running to point people in the right
>> > direction.
>>
>> Well, but it seems (recently?) that a policy has been put into place
>> that POMs already in the repository should not be corrected or
>> improved, in order to preserve repeatability for builds depending on
>> the existing version, and that corrections should be done by making
>> new releases.  It's hard enough to get projects to care about
>> providing Maven POMs, but to ask for a new release seems a bit much.
>
> There's always the possibiblity of adding a new version ourselves with
> a build number, like 1.0-1 where only the pom changes. Only as an
> exception when a project is dead or don't release often

I think the process is somewhat broken and that the maven team is  
being far too strict about changing broken poms that were in fact  
installed by the maven team, not supplied by the project.  (xmlbeans  
is the case in point for me).  I also think that traceability of  
where poms came from and under what auspices they are added to either  
the repository svn tree or ibiblio is sorely lacking.  (again  
xmlbeans being my sore spot).  We are in the situation where a pom  
appeared from an unknown source, is wrong, did not come from the  
project, and can't be updated.  Much as I like maven it's hard for me  
to see how this is going to lead to success.

thanks
david jencks

>
>>
>> It also may seem ideal to have projects take care of their own POMs,
>> but it makes it frustrating for users to provide information on  
>> fixes.
>>  I know, personally, I've cut down on contributing to central
>> repository improvement.  I've taken to simply installing new jars to
>> my internal repository, because asking individual projects to do it
>> gives slow-to-no returns.  I put top-level exclusions into
>> dependencyManagement rather than request changes to POMs, because
>> again, there seems no process for actually getting that to happen
>> that's not haphazard.  I'll try to work on doing better, but the
>> cost-reward ratio isn't helping.
>
> Well, if everybody does that then there's no community benefit and we
> all lose. When I need something not in ibiblio I put it there because
> it's not much harder than putting it in my local repo.
>
>>
>> I think maybe some either feature or convention for handling version
>> changes to just POMs so they can be improved without another release
>> of the software would help.  Some clarification/policy statements on
>> when I should go straight to the project responsible for a jar vs.
>> filing in Maven evangelism for uploads & for POM improvements  
>> might be
>> helpful.  Certainly some of the 2.1 planned features (like being able
>> to rely on geronimo-transaction & have that take care of anything
>> relying in javax.transaction:jta...) could help.  I think some  
>> concept
>> work needs to go into optional dependencies, because it we can't
>> control when Spring decides they want to stop providing modularized
>> jars, and move to a single jar that will essentially have all  
>> optional
>> dependencies.  I'm not looking forward to getting my projects to work
>> with Spring 2.0.
>
> You can always have poms for different maven configurations against
> only one jar, see
> http://jroller.com/page/carlossg?entry=optional_dependencies_in_maven
>
>>
>> Believe me, this is all coming from someone who's been trying.  I've
>> filed bug reports with Spring, and Lucene, other projects to get  
>> Maven
>> uploads.  I've volunteered to work on providing and maintaining a
>> Maven 2 build for an incubator project so that it will be easy to
>> provide Maven jars & poms when the time comes.
>
> You can see in http://opensource.atlassian.com/projects/spring/ 
> browse/SPR-1484
> that Spring got pressure from their users to provide POMs, it's by far
> the most popular issue. Any project that wants to succeed must listen
> to their users
>
>>
>> Oh, another quality issue.  -source and -javadoc jars.  It really
>> slows down running eclipse:eclipse when half or more of my
>> dependencies don't have these jars.  And a lot don't.  All of
>> spring-1.2.7, for instance.
>
> You can download Spring, jar the stuff up and submit them for adding
> to ibiblio. I don't understand how can you complain when maven is the
> only project providing that feature.
>
>>
>> My purpose isn't just to complain.  I just think that there's  
>> going to
>> have be more to it than "it'll get better over time" for the central
>> repository to improve, because, from the narrow view of the things I
>> use, it's getting a bit worse, not better right now.
>
> My view is that it's getting better, more jars, more poms, more
> javadocs, more sources, and more and more projects caring about the
> repository, being a must for them.
>
>>
>> --
>> Stephen Duncan Jr
>> www.stephenduncanjr.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@...
>> For additional commands, e-mail: dev-help@...
>>
>>
>
>
> --
> I could give you my word as a Spaniard.
> No good. I've known too many Spaniards.
>                             -- The Princess Bride
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[jrduncans]

On 7/4/06, Carlos Sanchez <carlos@...> wrote:

> On 7/5/06, Stephen Duncan <stephen.duncan@...> wrote:
> > On 7/4/06, Jason van Zyl <jason@...> wrote:
> > >
> > > On 4 Jul 06, at 2:37 PM 4 Jul 06, Steve Loughran wrote:
> > >
> > > The metadata will never be perfect but right now I still
> > > think it's far from being ideal because we have no real active
> > > process of improving it on a large scale. Carlos puts in a _lot_ of
> > > time trying to correct things and absorb changes submitted for
> > > improvement but as mentioned in the previous message it's a matter of
> > > education and automated tools running to point people in the right
> > > direction.
> >
> > Well, but it seems (recently?) that a policy has been put into place
> > that POMs already in the repository should not be corrected or
> > improved, in order to preserve repeatability for builds depending on
> > the existing version, and that corrections should be done by making
> > new releases.  It's hard enough to get projects to care about
> > providing Maven POMs, but to ask for a new release seems a bit much.
>
> There's always the possibiblity of adding a new version ourselves with
> a build number, like 1.0-1 where only the pom changes. Only as an
> exception when a project is dead or don't release often

That's essentially what I suggesting later on as an option for
by-convention fix for allowing fixes to POMs.  I don't think I agree
that it's an exceptional case though.   What do you see as the correct
process here?  That the issue for POM improvement be placed into the
other projects issue tracker, not Maven Evangelism, and that the
project make a new release with the changes?  Is that likely to
actually happen?

> >
> > It also may seem ideal to have projects take care of their own POMs,
> > but it makes it frustrating for users to provide information on fixes.
> >  I know, personally, I've cut down on contributing to central
> > repository improvement.  I've taken to simply installing new jars to
> > my internal repository, because asking individual projects to do it
> > gives slow-to-no returns.  I put top-level exclusions into
> > dependencyManagement rather than request changes to POMs, because
> > again, there seems no process for actually getting that to happen
> > that's not haphazard.  I'll try to work on doing better, but the
> > cost-reward ratio isn't helping.
>
> Well, if everybody does that then there's no community benefit and we
> all lose. When I need something not in ibiblio I put it there because
> it's not much harder than putting it in my local repo.

Like I said, I know I shouldn't do that.  But it certainly doesn't
feel as easy to get it put into ibiblio.  mvn deploy:deploy-file for
myself seems much easier than getting someone else's stuff into
ibiblio, where I need to make sure of all the details for groupId,
license, the rest of the POM is accurate, etc.  Plus, it's not even
clear that doing so for a third-party app was the encouraged way to
go, vs. trying to get the third-party to do it themselves.  If I
should be doing it, I guess I'll work to make an upload bundle for
tagsoup 1.0 tomorrow...

> > I think maybe some either feature or convention for handling version
> > changes to just POMs so they can be improved without another release
> > of the software would help.  Some clarification/policy statements on
> > when I should go straight to the project responsible for a jar vs.
> > filing in Maven evangelism for uploads & for POM improvements might be
> > helpful.  Certainly some of the 2.1 planned features (like being able
> > to rely on geronimo-transaction & have that take care of anything
> > relying in javax.transaction:jta...) could help.  I think some concept
> > work needs to go into optional dependencies, because it we can't
> > control when Spring decides they want to stop providing modularized
> > jars, and move to a single jar that will essentially have all optional
> > dependencies.  I'm not looking forward to getting my projects to work
> > with Spring 2.0.
>
> You can always have poms for different maven configurations against
> only one jar, see
> http://jroller.com/page/carlossg?entry=optional_dependencies_in_maven
>
> >
> > Believe me, this is all coming from someone who's been trying.  I've
> > filed bug reports with Spring, and Lucene, other projects to get Maven
> > uploads.  I've volunteered to work on providing and maintaining a
> > Maven 2 build for an incubator project so that it will be easy to
> > provide Maven jars & poms when the time comes.
>
> You can see in http://opensource.atlassian.com/projects/spring/browse/SPR-1484
> that Spring got pressure from their users to provide POMs, it's by far
> the most popular issue. Any project that wants to succeed must listen
> to their users

Again, it doesn't feel like that means much.  I had to work with you &
another helpful Maven user to get POMs for Spring 1.2.7.  And there
are no source or javadoc jars there.  Spring 1.2.8 doesn't have POMs,
and has been there for over a month:
http://ibiblio.org/maven2/org/springframework/spring-support/1.2.8/

> >
> > Oh, another quality issue.  -source and -javadoc jars.  It really
> > slows down running eclipse:eclipse when half or more of my
> > dependencies don't have these jars.  And a lot don't.  All of
> > spring-1.2.7, for instance.
>
> You can download Spring, jar the stuff up and submit them for adding
> to ibiblio. I don't understand how can you complain when maven is the
> only project providing that feature.

I apologize, but I don't understand what you were saying here.

> >
> > My purpose isn't just to complain.  I just think that there's going to
> > have be more to it than "it'll get better over time" for the central
> > repository to improve, because, from the narrow view of the things I
> > use, it's getting a bit worse, not better right now.
>
> My view is that it's getting better, more jars, more poms, more
> javadocs, more sources, and more and more projects caring about the
> repository, being a must for them.
>
> >
> > --
> > Stephen Duncan Jr
> > www.stephenduncanjr.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@...
> > For additional commands, e-mail: dev-help@...
> >
> >
>
>
> --
> I could give you my word as a Spaniard.
> No good. I've known too many Spaniards.
>                              -- The Princess Bride
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>

--
Stephen Duncan Jr
www.stephenduncanjr.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[brettporter]

On 5/07/2006 10:54 AM, David Jencks wrote:
> I think the process is somewhat broken and that the maven team is being
> far too strict about changing broken poms that were in fact installed by
> the maven team, not supplied by the project.  (xmlbeans is the case in
> point for me).  I also think that traceability of where poms came from
> and under what auspices they are added to either the repository svn tree
> or ibiblio is sorely lacking.  (again xmlbeans being my sore spot).  We
> are in the situation where a pom appeared from an unknown source, is
> wrong, did not come from the project, and can't be updated.  Much as I
> like maven it's hard for me to see how this is going to lead to success.

David,

I thought we'd agreed to fix that up? I can't find the issue on it other
than http://jira.codehaus.org/browse/MEV-406 which is not the one I
recall (that one is about stax which is a whole other problem beyond the
scope of Maven too).

- Brett

--
Brett Porter <brett@...>
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Jörg Schaible]

Brett Porter wrote on Monday, July 03, 2006 2:27 PM:

> The original snapshot feature works just fine.
>
> There was a particular variation of the feature added that
> doesn't work
> as designed. (MNG-1908). The variation works exactly the same way but
> reuses the file on the server.
>
> Using uniqueVersion = false (the default) and a clean up
> script on the
> server is a workaround.

C'mon. The opposite behaviour was the default for Maven 1, where this worked perfectly. And using uniqueVersion == false did not work out very well when releasing multi-module projects (don't know if it works now).

This is one of Maven's core functionalitioes htat is currently not working in M2!

- Jörg

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[brettporter]

On 5/07/2006 3:51 PM, Jörg Schaible wrote:
> C'mon. The opposite behaviour was the default for Maven 1, where this worked perfectly.

And everyone complained about how slow it was. As I said, there is a
working fix in MNG-1908, but it is not in the release because the
performance is, IMO, unacceptable.

> And using uniqueVersion == false did not work out very well when releasing multi-module projects (don't know if it works now).

It does in both versions.

- Brett

--
Brett Porter <brett@...>
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Ralph Goers]

Carlos Sanchez wrote:
>
> Yes you can, it's not the best way to do it but you can, by adding
> explicitly the dependency with the versoin you want to your pom. In
> the very worst case you have to add all transitive deendencies to your
> pom, like in Maven 1.
That is so impractical as to be nonsensical. Our Configuration
Management folks require that all projects use the same dependencies
from the top to the bottom. For example, we build our base set of
frameworks with one multiproject build, then a higher level of
frameworks, and then finally the product itself. Each of these must be
built and unit tested with the same third party jars. Furthermore, the
product can consist of a war and one or more ejb's which may all be
packaged in an ear. These must also all have all the same versions of
the jars. The solution you propose makes that tedious, error prone, and
would require our CM folks to manually inspect every pom to insure
nothing is done improperly. At least with Maven 1 we can have a
build.properties that all projects share.  In our case, the ideal
solution in Maven 2 would be to have a "master" pom with nothing in it
but a dependencyManagement section (with something like override="true"
set as an attribute) that all our projects would reference.

Ralph

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[cziegeler]

Brett Porter wrote:
>
> The first thing I'd suggest is for those having problems to try another
> mirror. I know requiring everyone to do that is a pain and not a long
> term solution, but I'd like to see how much that reduces the problem.
>
I'm not sure if the following problem has already been mentioned in this
thread as somehow my mail client messed up :(

We experience problems like the build is working on one machine while
not on another one. The reason for this is changing poms on the public
repository! M2 does not recognize when a pom is changed on the
repository, so once you have a pom for a dependency it's never updated.
If now someone builds before the update he has the old version while
someone who builds after the update for the first time, gets the new
version.
And this is exactly the reason why on some machines our build works
(they have the new version of a dependency which brings in other
transitive dependencies). It took us some time to recognize this as one
of the main problems.

Obviously, a pom in a public repository should never change, but as
there are faulty poms out there, these need to be fixed. So its very
likely that this will still happen in the future.
So it would be good if m2 could detect changes in the pom and then
update the local version. This would at least solve most of our problems.

I'm still thinking that adding a switch "turn off transitive dependency
handling" would be a nice think as then you really control what you get
without adding excludes all over the place.

And it seems that m2 is trying to get poms for snapshots even in offline
mode (haven't checked if this is filed as a bug already) and more
important it downloads snapshot versions from the repository although a
newer version of the snapshot is in the local repository. And the
interesting thing is, that although a version has been downloaded, the
local one is later on used.

Carsten
--
Carsten Ziegeler - Open Source Group, S&N AG
http://www.s-und-n.de
http://www.osoco.org/weblogs/rael/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Ralph Goers]

Brett Porter wrote:

> It depends on how you use them as to the best solution here. I assume
> that they are customised for cocoon, so they shouldn't be considered
> to be the same as the original. In that case, I'd suggest you release
> them under your own groupID (maybe org.apache.cocoon.thirdparty) so
> that they don't conflict (bearing in mind that someone will
> transitively receive both that and the original if they are both
> used). I'm assuming this isn't the case as you are only bundling these
> modified versions into a distributable app, not as part of a reusable
> component?
>
> Hope this helps,
> Brett
>

Actually, I'm not exactly sure why non-released jars are currently
included, but I'm almost certain that in most cases it is not because it
is customized for Cocoon. In looking in our latest 2.1.9 release (which
isn't built with Maven) I see that jcs, chaperon, jackrabbit, deli,
dojo, jing, joost, myfaces, poi, wsrp4j, xmldb and xreporter were all
included in the release from some interim version that is probably
unknown to those projects.  I suspect for many of them this was done
because they included critical fixes.  Since Cocoon contains so many
third party jars we would either have to make releases knowing that some
things are broken or never put out a release.

However, your point about naming them o.a.c.thirdparty is well taken (at
least by me).  A complaint I frequently have had is that it is often
difficult to locate the exact source that was used to create the jar.
Sometimes they are named with the svn revision with makes it easy.
Sometimes they have a datestamp, which isn't necessarily guaranteed to
get you the exact source, and sometimes they have a name like
wsrp4j-shared-0.3-dev.jar (wsrp4j is in inclubator and I don't believe
it has ever done a release, but this version number is really bad since
there is no tag in svn matching 0.3-dev that I can see). In wsrp4j's
case, I'm fairly sure Cocoon's usage of it is considered experimental -
it gives users a way to have early access to it for experimentation
purposes.

Ralph

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[brettporter]

Thanks Carsten. The first part was certainly already discussed (so you
can see it in the mail archives).

It'd be good if you could file bugs for the last 3 things and we can
schedule them for upcoming releases.

- Brett

On 5/07/2006 4:45 PM, Carsten Ziegeler wrote:

> Brett Porter wrote:
>> The first thing I'd suggest is for those having problems to try another
>> mirror. I know requiring everyone to do that is a pain and not a long
>> term solution, but I'd like to see how much that reduces the problem.
>>
> I'm not sure if the following problem has already been mentioned in this
> thread as somehow my mail client messed up :(
>
> We experience problems like the build is working on one machine while
> not on another one. The reason for this is changing poms on the public
> repository! M2 does not recognize when a pom is changed on the
> repository, so once you have a pom for a dependency it's never updated.
> If now someone builds before the update he has the old version while
> someone who builds after the update for the first time, gets the new
> version.
> And this is exactly the reason why on some machines our build works
> (they have the new version of a dependency which brings in other
> transitive dependencies). It took us some time to recognize this as one
> of the main problems.
>
> Obviously, a pom in a public repository should never change, but as
> there are faulty poms out there, these need to be fixed. So its very
> likely that this will still happen in the future.
> So it would be good if m2 could detect changes in the pom and then
> update the local version. This would at least solve most of our problems.
>
> I'm still thinking that adding a switch "turn off transitive dependency
> handling" would be a nice think as then you really control what you get
> without adding excludes all over the place.
>
> And it seems that m2 is trying to get poms for snapshots even in offline
> mode (haven't checked if this is filed as a bug already) and more
> important it downloads snapshot versions from the repository although a
> newer version of the snapshot is in the local repository. And the
> interesting thing is, that although a version has been downloaded, the
> local one is later on used.
>
> Carsten

--
Brett Porter <brett@...>
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[djencks]

On Jul 4, 2006, at 6:33 PM, Brett Porter wrote:

> On 5/07/2006 10:54 AM, David Jencks wrote:
>> I think the process is somewhat broken and that the maven team is  
>> being far too strict about changing broken poms that were in fact  
>> installed by the maven team, not supplied by the project.  
>> (xmlbeans is the case in point for me).  I also think that  
>> traceability of where poms came from and under what auspices they  
>> are added to either the repository svn tree or ibiblio is sorely  
>> lacking.  (again xmlbeans being my sore spot).  We are in the  
>> situation where a pom appeared from an unknown source, is wrong,  
>> did not come from the project, and can't be updated.  Much as I  
>> like maven it's hard for me to see how this is going to lead to  
>> success.
>
> David,
>
> I thought we'd agreed to fix that up? I can't find the issue on it  
> other than http://jira.codehaus.org/browse/MEV-406 which is not the  
> one I recall (that one is about stax which is a whole other problem  
> beyond the scope of Maven too).

that's the one.  The xmlbeans pom does not have the required stax-api  
dependency listed in it.  As that issue notes, Carlos declined to  
change the existing pom.  IIUC Carlos said I needed to get xmlbeans  
to produce the pom, which I am currently working on, see http://
issues.apache.org/jira/browse/XMLBEANS-277

The xmlbeans team have agreed in principle to cooperate but have not  
in fact yet reviewed my first draft for things like correct urls.  I  
originally planned to produce a complete set of poms for their 3 jars  
and 3 releases but am not sure how to test one of them.    To me it  
seems like its turned into an enormous amount of work to fix a really  
obvious problem.  One of the most frustrating things for me is that  
there is no way to find out where the existing wrong poms came from.  
Second most frustrating was that the MEV instructions turned out to  
be 100% inaccurate, so following them to the letter was a complete  
waste of time.

There was another problem with the maven xmlbeans plugin that Kris  
Bravo fixed immediately.

thanks
david jencks

>
> - Brett
>
> --
> Brett Porter <brett@...>
> Apache Maven - http://maven.apache.org/
> Better Builds with Maven - http://library.mergere.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[brettporter]

Hi Ralph,

You've got a general versioning problem here, and you'll find the answer
to "how do I do this with Maven?" will be more straightforward once
considering the question of how you should generally deal with them. As
you've said, this is already a problem for you as you don't know where
they really came from.

Ignoring Maven for a moment, there's a couple of questions I'd consider
(bear in mind I'm not a Cocoon user so I may be misunderstanding how
they eventually get used):
- how would a user that used one of these dependencies themselves in
addition to that Cocoon component select which version of the dependency
to use? Would they use the Cocoon-supplied one, one they select, or both?
- are the changes Cocoon specific? Will you always be using a custom
release, or will you pick up the next release when it is available?

There are a couple of options for addressing this use case in Maven.
- include the JARs in SVN, and reference it as an additional repository
file://localhost/${basedir}/extra-jar-repo

- host your own repository of these artifacts (this is basically
equivalent to the above and probably easier to work with, and could be
accommodated in a general ASF repository of such things, once taking
into account the licensing guidelines)

- ask projects to do a beta/point release for you (it should be easy to
make a case for this if the version you are using is both stable and
contains critical fixes)

- "fork" the code (taking into consideration licensing guidelines)
either temporarily or permanently in your own repository (ie
http://svnbook.red-bean.com/nightly/en/svn.advanced.vendorbr.html). Make
it part of your build, and do the custom group ID thing. We really need
the "provides" functionality planned for Maven 2.1 to make this feasible
if it is intended to be a drop in replacement of the original JAR (so
that you don't get duplicates in the dependency chain), but it can
certainly work.

BTW, I also think we need specific support for this type of artifact (it
is essentially a "vendor" release), and it is under consideration for
Maven 2.1.

Your point about never being a Cocoon release while waiting for
dependencies to release is something I'm familiar with and there's
definitely a need for this. However, is it possible to reduce the impact
of it by being able to release individual components at different times
so that they can track the release cycles of their dependencies? This
sounds very similar to the scenario of Maven plugins, where it works
quite successfully.

HTH,
Brett

On 5/07/2006 4:49 PM, Ralph Goers wrote:

> Brett Porter wrote:
>> It depends on how you use them as to the best solution here. I assume
>> that they are customised for cocoon, so they shouldn't be considered
>> to be the same as the original. In that case, I'd suggest you release
>> them under your own groupID (maybe org.apache.cocoon.thirdparty) so
>> that they don't conflict (bearing in mind that someone will
>> transitively receive both that and the original if they are both
>> used). I'm assuming this isn't the case as you are only bundling these
>> modified versions into a distributable app, not as part of a reusable
>> component?
>>
>> Hope this helps,
>> Brett
>>
> Actually, I'm not exactly sure why non-released jars are currently
> included, but I'm almost certain that in most cases it is not because it
> is customized for Cocoon. In looking in our latest 2.1.9 release (which
> isn't built with Maven) I see that jcs, chaperon, jackrabbit, deli,
> dojo, jing, joost, myfaces, poi, wsrp4j, xmldb and xreporter were all
> included in the release from some interim version that is probably
> unknown to those projects.  I suspect for many of them this was done
> because they included critical fixes.  Since Cocoon contains so many
> third party jars we would either have to make releases knowing that some
> things are broken or never put out a release.
>
> However, your point about naming them o.a.c.thirdparty is well taken (at
> least by me).  A complaint I frequently have had is that it is often
> difficult to locate the exact source that was used to create the jar.
> Sometimes they are named with the svn revision with makes it easy.
> Sometimes they have a datestamp, which isn't necessarily guaranteed to
> get you the exact source, and sometimes they have a name like
> wsrp4j-shared-0.3-dev.jar (wsrp4j is in inclubator and I don't believe
> it has ever done a release, but this version number is really bad since
> there is no tag in svn matching 0.3-dev that I can see). In wsrp4j's
> case, I'm fairly sure Cocoon's usage of it is considered experimental -
> it gives users a way to have early access to it for experimentation
> purposes.
>
> Ralph
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

--
Brett Porter <brett@...>
Apache Maven - http://maven.apache.org/
Better Builds with Maven - http://library.mergere.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Geoffrey De Smet]

Brett Porter wrote:
> Hi Ralph,
>
> There are a couple of options for addressing this use case in Maven.
> - include the JARs in SVN, and reference it as an additional repository
> file://localhost/${basedir}/extra-jar-repo
>
>
When you declare such a repository in your parent pom, the child poms
don't use the parent's basedir, but their own basedir. Is there a
workaround for that problem?


--
With kind regards,
Geoffrey De Smet


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...