sources as conf or type

[Stephen Haberman]

Hi,

For pom->ivy translation, it seems like a separate "sources" conf is not the best approach.

There are three reasons:

1) It overlaps with the type=jar|source functionality.

The master conf could have both the jar and source artifacts in it. If people only want the jars, they could add "type=jar" limitation to their resolver, if they want both jar and source, they could just resolve all types.

2) Separating source artifacts from the master conf hurts transitive source retrieval.

As it is now, if a project you depend on only maps their dependencies as master->master instead of master->master;sources->sources, then you're stuck with their decision of source artifacts not being important and cannot transitively retrieve them.

3) When multiple confs do exist, it makes source artifact placement arbitrary.

E.g. if I have conf A, and conf B, with a.jar, a-src.jar, b.jar, and b-src.jar, what conf do I put a-src.jar and b-src.jar in? If I follow the separate "sources" conf convention, they either both go in "sources" (but then people that only want conf A's a.jar+a-src.jar will also get the b-src.jar), or I make separate "a-sources" and "b-sources" confs, which leads to lots of extra confs and ugly downstream mappings (e.g. master->a,a-sources).

Putting source artifacts in the same conf as their corresponding jar artifact seems to solve all of these problems, while keeping the ability to distinguish type=jar|source and only download the artifacts you are interested in as an orthogonal concern.

Is it too late in the game to change this? Am I missing something obvious?

Thanks,
Stephen

[Mitch Gitman]

I think it's fair to say that the established best practice is to NOT make a
separate Ivy conf for things like source or Javadoc.

For one thing, it becomes a transitive dependency nightmare to keep on
having to create a source conf in your Ivy module and then have your source
conf depend on its dependencies' source Ivy confs--which means you need to
add source Ivy confs to those Ivy modules, and so on and so on.

For another thing, IvyDE will only automatically locate and open the source
for a dependency in Eclipse if the source is in the same conf as the binary
in question. In addition, the source artifact's name should have one of the
suffixes configured in IvyDE, like -source or -src.

What this means is that the ivy.xml generated by the automated pom->Ivy
translation should be viewed very much as an intermediate artifact to be
manually massaged rather than a finished product. I'm not sure the extra
smarts involved in that manual massaging should get incorporated into the
purview of that automated translation since the translation is trying to be
literal.

On Fri, Oct 23, 2009 at 8:26 AM, stephenh <stephen@...> wrote:

>
> Hi,
>
> For pom->ivy translation, it seems like a separate "sources" conf is not
> the
> best approach.
>
> There are three reasons:
>
> 1) It overlaps with the type=jar|source functionality.
>
> The master conf could have both the jar and source artifacts in it. If
> people only want the jars, they could add "type=jar" limitation to their
> resolver, if they want both jar and source, they could just resolve all
> types.
>
> 2) Separating source artifacts from the master conf hurts transitive source
> retrieval.
>
> As it is now, if a project you depend on only maps their dependencies as
> master->master instead of master->master;sources->sources, then you're
> stuck
> with their decision of source artifacts not being important and cannot
> transitively retrieve them.
>
> 3) When multiple confs do exist, it makes source artifact placement
> arbitrary.
>
> E.g. if I have conf A, and conf B, with a.jar, a-src.jar, b.jar, and
> b-src.jar, what conf do I put a-src.jar and b-src.jar in? If I follow the
> separate "sources" conf convention, they either both go in "sources" (but
> then people that only want conf A's a.jar+a-src.jar will also get the
> b-src.jar), or I make separate "a-sources" and "b-sources" confs, which
> leads to lots of extra confs and ugly downstream mappings (e.g.
> master->a,a-sources).
>
> Putting source artifacts in the same conf as their corresponding jar
> artifact seems to solve all of these problems, while keeping the ability to
> distinguish type=jar|source and only download the artifacts you are
> interested in as an orthogonal concern.
>
> Is it too late in the game to change this? Am I missing something obvious?
>
> Thanks,
> Stephen
>
> --
> View this message in context:
> http://www.nabble.com/sources-as-conf-or-type-tp26028446p26028446.html
> Sent from the ivy-user mailing list archive at Nabble.com.
>
>

[Niklas Matthies-2]

On Wed 2009-10-28 at 13:29h, Mitch Gitman wrote on ivy-user:
> I think it's fair to say that the established best practice is to NOT make a
> separate Ivy conf for things like source or Javadoc.
>
> For one thing, it becomes a transitive dependency nightmare to keep on
> having to create a source conf in your Ivy module and then have your source
> conf depend on its dependencies' source Ivy confs--which means you need to
> add source Ivy confs to those Ivy modules, and so on and so on.

Not sure I agree. You generally have to add a bunch of configurations
anyway (compile, runtime, api, test, javadoc, ...). For our in-house
modules, we have an XML file defining the common configurations and
mappings (it resides along the common build files which are initially
retrieved via Ivy as well), and just <include> that file into each
ivy.xml. And the mapping for the source configurations is just
source->source(default), no difficulties there.

Using just filetype, on the other hand, complicates things if you have
programmatic artifacts other than jar (ear, dll, ...). Then you'd have
to figure out the right list of types to retrieve all required
artifacts.

If I understand correctly, you always specify type="jar" when
retrieving programmatic artifacts? Or do you always retrieve the
source along with it, even when packaging?

And where do you put javadocs?

-- Niklas Matthies

[Mitch Gitman]

Answers inline prefaced by MG:

On Wed, Oct 28, 2009 at 2:15 PM, Niklas Matthies <ml_ivy-user@...>wrote:

> On Wed 2009-10-28 at 13:29h, Mitch Gitman wrote on ivy-user:
> > I think it's fair to say that the established best practice is to NOT
> make a
> > separate Ivy conf for things like source or Javadoc.
> >
> > For one thing, it becomes a transitive dependency nightmare to keep on
> > having to create a source conf in your Ivy module and then have your
> source
> > conf depend on its dependencies' source Ivy confs--which means you need
> to
> > add source Ivy confs to those Ivy modules, and so on and so on.
>
> Not sure I agree. You generally have to add a bunch of configurations
> anyway (compile, runtime, api, test, javadoc, ...). For our in-house
> modules, we have an XML file defining the common configurations and
> mappings (it resides along the common build files which are initially
> retrieved via Ivy as well), and just <include> that file into each
> ivy.xml. And the mapping for the source configurations is just
> source->source(default), no difficulties there.
>
> MG: Niklas, to address your particular point, I can tell you that I used to

follow the "source conf" approach myself. And here's one critical drawback I
came across. Sure, one can stick an include in the configurations section of
every ivy.xml and have that included set of confs have a source conf. That
way, you don't have to remember to add the source conf every time. But
you're still going to have to add the mapping again and again to the
dependencies:
source->source OR source->source(default)

It's easy enough to forget to add this. Niklas, I'm not sure what you mean
by "mappings" when you write: "For our in-house modules, we have an XML file
defining the common configurations and mappings."

But beyond that, even if there is a way to avoid requiring the dev to
manually add the same mapping again and again, specifying the mapping is
still a case of putting protocol in front of essence. You're not adding much
semantically the way you are when you're saying the "test" conf needs these
particular dependencies while the "api" conf doesn't.

And this gets to the central argument against special source confs, and it's
a philosophical one. Suppose your Ivy module is publishing different binary
artifacts, one artifact or one combination for each conf. If you have confs
"a" and "b" and "c", do you then come up with "a-source" and b-source" and
"c-source" confs, and so on? Eventually, this starts smelling like an
anti-pattern.

A better way of thinking of source and Javadoc is that they are just a
couple other ways of representing the same "thing," just like the binary
with the .class files is a way of representing that "thing." It's the actual
combination of classes and resources that distinguishes the conf and its
deliverables, not whether it's represented at that particular time as a
binary or source or API docs.

> Using just filetype, on the other hand, complicates things if you have
> programmatic artifacts other than jar (ear, dll, ...). Then you'd have
> to figure out the right list of types to retrieve all required
> artifacts.
>
> If I understand correctly, you always specify type="jar" when
> retrieving programmatic artifacts? Or do you always retrieve the
> source along with it, even when packaging?
>
> MG: I agree with you on this score. The one drawback I've discovered about

putting binary and source and generated doc in the same conf is that you'll
wind up with unwanted artifacts in your deployment. This is why I made an
argument for an excludetype attribute in the *ivy:cachefileset* task. See my
message in this thread:
http://www.nabble.com/IvyDE-automatically-include-javadoc-and-source-td24746732.html

Then you could say excludetype="source,javadoc" if you know these are the
confs to avoid to avoid distributing the source and Javadoc. (Or hey, maybe
you do want the source in there.)

Now that I've had a chance to think more carefully about this problem, I
realize I could make a much more compelling argument in favor of that
feature than I did at the time, but I'll save that argument for another day.

And where do you put javadocs?
>
> -- Niklas Matthies
>

MG: P.S. There's one assumption I've been working from when it comes to
IvyDE, and I'm confident someone will be kind enough to correct me if I'm
wrong in this assumption. It's that IvyDE will only allow Eclipse to
automatically open the source for dependencies *if the source belongs to the
same Ivy conf as the associated binary*.

[Niklas Matthies-2]

On Wed 2009-10-28 at 19:52h, Mitch Gitman wrote on ivy-user:
> On Wed, Oct 28, 2009 at 2:15 PM, Niklas Matthies wrote:
:

> MG: Niklas, to address your particular point, I can tell you that I
> used to follow the "source conf" approach myself. And here's one
> critical drawback I came across. Sure, one can stick an include in
> the configurations section of every ivy.xml and have that included
> set of confs have a source conf. That way, you don't have to
> remember to add the source conf every time. But you're still going
> to have to add the mapping again and again to the dependencies:
> source->source OR source->source(default)
>
> It's easy enough to forget to add this. Niklas, I'm not sure what
> you mean by "mappings" when you write: "For our in-house modules, we
> have an XML file defining the common configurations and mappings."

I'm talking about the defaultconfmapping attribute of the
configurations element contained in the included file. This way you
don't have to add source->source on every single dependency.

> But beyond that, even if there is a way to avoid requiring the dev
> to manually add the same mapping again and again, specifying the
> mapping is still a case of putting protocol in front of essence.
> You're not adding much semantically the way you are when you're
> saying the "test" conf needs these particular dependencies while the
> "api" conf doesn't.

Well, it depends (no pun). With source->source you specify that the
dependencies use the same naming convention for the source artifact
configuration as the current module. It's really not different from
api->api in that respect. Either you somehow force all modules
(including third-party ones) to all use the same convention, or else
you need to (be able to) specify a mapping.

When using just type="source" (or is it "src"?), you just presume that
all modules involved use the convention to put source artifacts in the
"regular" configurations (and also mark them with type="source" and
not some other type), and a module doesn't have the chance to specify
that its source artifacts actually reside in a different conf.

> And this gets to the central argument against special source confs,
> and it's a philosophical one. Suppose your Ivy module is publishing
> different binary artifacts, one artifact or one combination for each
> conf. If you have confs "a" and "b" and "c", do you then come up
> with "a-source" and b-source" and "c-source" confs, and so on?
> Eventually, this starts smelling like an anti-pattern.

We don't really have that case, at least as far as the module's own
sources are concerned. Different sources mean different modules. All
artifacts of a module are built from the same set of sources that
belong together, as they are versioned together. I can see how it
could be useful for the sources of the dependencies, to reduce the
number of sources to download, but so far we havn't had a problem with
that. In cases were it would become a problem, then yes I imagine we
might define multiple source confs. Is that your motivation, not
having to download too many dependent sources?

> A better way of thinking of source and Javadoc is that they are just
> a couple other ways of representing the same "thing," just like the
> binary with the .class files is a way of representing that "thing."
> It's the actual combination of classes and resources that
> distinguishes the conf and its deliverables, not whether it's
> represented at that particular time as a binary or source or API
> docs.

I can follow that point. But are your "things" so different that they
have significantly different APIs, for example? My guess is that we'd
place sufficiently different "things" in different modules.

:
> Then you could say excludetype="source,javadoc" if you know these are the
> confs to avoid to avoid distributing the source and Javadoc. (Or hey, maybe
> you do want the source in there.)

Yeah, but in the end it's just the same problem backwards, in that it
really is an open-ended list. Maybe you actually need
excludetype="source,javadoc,pdfmanual,debugsymbols" or whatever.

In any case, it's an interesting discussion, as both approaches have
drawbacks, and it isn't perfect either way.

:
> And where do you put javadocs?

In a javadoc conf. We also generate interlinked javadocs to a network
drive location. Meaning that given an API dependency B;1.1->A;1.5,
when publishing A 1.5, its javadocs are published so some directory
docs/A/1.5/, and then when publishing B 1.1. it publishes docs to
docs/B/1.1/ that link to the A 1.5 docs wherever they reference A.
So over time we build a versioned javadoc repository that way that we
can directly browse.

> MG: P.S. There's one assumption I've been working from when it comes to
> IvyDE, and I'm confident someone will be kind enough to correct me if I'm
> wrong in this assumption. It's that IvyDE will only allow Eclipse to
> automatically open the source for dependencies *if the source belongs to the
> same Ivy conf as the associated binary*.

I use NetBeans, so... :)

-- Niklas Matthies

[Stephen Haberman]

Mitch Gitman wrote:

MG: P.S. There's one assumption I've been working from when it comes to
IvyDE, and I'm confident someone will be kind enough to correct me if I'm
wrong in this assumption. It's that IvyDE will only allow Eclipse to
automatically open the source for dependencies *if the source belongs to the
same Ivy conf as the associated binary*.

Perhaps IvyDE used to have that limitation, but currently I have a 3rd party library with separate default/sources confs and it is hooking up the source just fine.

- Stephen

[Stephen Haberman]

Niklas Matthies-2 wrote:

Yeah, but in the end it's just the same problem backwards, in that it
really is an open-ended list. Maybe you actually need
excludetype="source,javadoc,pdfmanual,debugsymbols" or whatever.

In any case, it's an interesting discussion, as both approaches have
drawbacks, and it isn't perfect either way.

Let me know if I'm misinterpreting here, but your main point seems to be that using same conf with type=jar|source is not ideal because type is a free form field and upstream projects could use different values. So you'd need a convention.

Which is true. But isn't that already the same with the conf names?

E.g. you say "okay, what are our repo's conf names" and/or "okay, what are our repo's type names" and once you've answered that, aren't separate-conf or same-conf basically the same?

And if so, doesn't same-conf, with its simplification of transitive dependencies (e.g. no dummy sources->sources to worry about), edge out a win?

Nonetheless, I like that Ivy can do both, I would not want to change that.

What annoys me specifically is the maven pom->ivy working this way, and giving me an ivy.xml file like:

http://sprunge.us/PgAa

Where I'll get the jetty source but not the jetty-util source because they forgot the "sources->sources(*)" mapping.

Also specifically for pom->ivy translation, I don't think making up a new sources conf makes sense, because the confs are primarily a mapping of Maven's scopes:

http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope

And so it would be more true to the original pom to leave out sources and javadoc confs, put the source & javadoc artifacts in the master conf and use type=jar|source|javadoc.

Yes, this is a convention, I just think its a better convention specifically for maven pom->ivy translation.

Ideally I'd like to see this changed in the pom->ivy code...any chance of that happening?

Thanks,
Stephen

[Niklas Matthies-2]

On Thu 2009-10-29 at 18:30h, Stephen Haberman wrote on ivy-user:

> Niklas Matthies-2 wrote:
> >
> > Yeah, but in the end it's just the same problem backwards, in that it
> > really is an open-ended list. Maybe you actually need
> > excludetype="source,javadoc,pdfmanual,debugsymbols" or whatever.
> >
> > In any case, it's an interesting discussion, as both approaches have
> > drawbacks, and it isn't perfect either way.
>
> Let me know if I'm misinterpreting here, but your main point seems
> to be that using same conf with type=jar|source is not ideal because
> type is a free form field and upstream projects could use different
> values. So you'd need a convention.
>
> Which is true. But isn't that already the same with the conf names?

Yes and no. For one, you can handle different conf name conventions
with conf mappings if necessary. For types on the other hand there's
nothing you can do, you'd be out of luck then. Secondly, confs are not
exclusive like types. Artifacts can belong to multiple confs, and you
can effectively define sub confs (or super confs) of other confs. On
the other hand you can't define sub types or super types. For example
you can't define a type "binary-deliverable" that would include jars,
ears, dlls and whatnot. This means that client modules can't just say
"gimme all binary deliverables", but instead need to know the explicit
list, and for *all* transitive dependencies at that.

If types were hierarchical nodes in a category graph, and a mapping
mechanism would be provided like for confs, then I would agree with
you. Types would be almost like a second type of confs then. Which I
think would be very useful.

:
> Where I'll get the jetty source but not the jetty-util source
> because they forgot the "sources->sources(*)" mapping.

Luckily there's dual resolvers. ;)
Yes, it's not ideal, but at least you *can* do something about it that
way.

-- Niklas Matthies

[Mitch Gitman]

Stephen, thanks for verifying this.

My apologies for sharing that erroneous information. I think I confused that
perceived--and nonexistent--limitation for the new strict rule introduced
with IvyDE 2.0.0 beta1 where the source artifacts have to be declared in the
ivy.xml to get picked up--regardless of whether they're in the same conf as
the associated binaries.

I would still go with the "make source a type" approach rather than the
"make source a conf" approach, but certainly the latter approach is
perfectly valid.

On Thu, Oct 29, 2009 at 5:38 PM, Stephen Haberman
<stephen@...>wrote:

>
>
>
> Mitch Gitman wrote:
> >
> > MG: P.S. There's one assumption I've been working from when it comes to
> > IvyDE, and I'm confident someone will be kind enough to correct me if I'm
> > wrong in this assumption. It's that IvyDE will only allow Eclipse to
> > automatically open the source for dependencies *if the source belongs to
> > the
> > same Ivy conf as the associated binary*.
> >
>
> Perhaps IvyDE used to have that limitation, but currently I have a 3rd
> party
> library with separate default/sources confs and it is hooking up the source
> just fine.
>
> - Stephen
>
> --
> View this message in context:
> http://www.nabble.com/sources-as-conf-or-type-tp26028446p26123274.html
> Sent from the ivy-user mailing list archive at Nabble.com.
>
>

[Stephen Haberman]

Niklas Matthies-2 wrote:

If types were hierarchical nodes in a category graph, and a mapping
mechanism would be provided like for confs, then I would agree with
you. Types would be almost like a second type of confs then. Which I
think would be very useful.

Hm...true, types are flat and non-mappable.

For me living mostly in a java/ibiblio world, the "all binaries" super type isn't a problem--since I haven't used ivy for non-java projects, I don't know how often the "any binary type even if I don't know about it ahead of time" thing would come up. Though I understand the gist of it.

Niklas Matthies-2 wrote:

> Where I'll get the jetty source but not the jetty-util source
> because they forgot the "sources->sources(*)" mapping.

Luckily there's dual resolvers. ;)
Yes, it's not ideal, but at least you *can* do something about it that
way.

Hehe, okay, technically yes, but yuck. The pom->ivy translation works well enough for me that I'd prefer to not maintain other projects' ivy files by hand just because of the non-transitive source.

Which I talked myself into it being a bug:

https://issues.apache.org/jira/browse/IVY-1137

- Stephen

[Stephen Haberman]

Mitch Gitman wrote:

Stephen, thanks for verifying this.

No problem.

Mitch Gitman wrote:

...for the new strict rule introduced with IvyDE 2.0.0 beta1 where the
source artifacts have to be declared in the ivy.xml to get picked up

I'm missing something--this rule makes sense to me. How did IvyDE previously go about finding source artifacts if they were not declared in the ivy.xml file?

Thanks,
Stephen

[Nicolas Lalevée]

Le 30 oct. 2009 à 04:43, Stephen Haberman a écrit :

>
>
> Niklas Matthies-2 wrote:
>>
>> If types were hierarchical nodes in a category graph, and a mapping
>> mechanism would be provided like for confs, then I would agree with
>> you. Types would be almost like a second type of confs then. Which I
>> think would be very useful.
>>
>
> Hm...true, types are flat and non-mappable.
>
> For me living mostly in a java/ibiblio world, the "all binaries"  
> super type
> isn't a problem--since I haven't used ivy for non-java projects, I  
> don't
> know how often the "any binary type even if I don't know about it  
> ahead of
> time" thing would come up. Though I understand the gist of it.
>
>
> Niklas Matthies-2 wrote:
>>
>>> Where I'll get the jetty source but not the jetty-util source
>>> because they forgot the "sources->sources(*)" mapping.
>>
>> Luckily there's dual resolvers. ;)
>> Yes, it's not ideal, but at least you *can* do something about it  
>> that
>> way.
>>
>
> Hehe, okay, technically yes, but yuck. The pom->ivy translation  
> works well
> enough for me that I'd prefer to not maintain other projects' ivy  
> files by
> hand just because of the non-transitive source.
>
> Which I talked myself into it being a bug:
>
> https://issues.apache.org/jira/browse/IVY-1137

Sorry I didn't have time to reply to this interesting discussion.  
Actually I already have that discussion around the same issue in pom /  
ivy mapping there:
http://old.nabble.com/pom2ivy-and-transitive-source-retrieving-to25112985.html#a25112985

I "just" (shame on me) forgot to actually implement the solution we  
agreed on. I'll do soon.

Nicolas

[Nicolas Lalevée]

Le 30 oct. 2009 à 04:56, Stephen Haberman a écrit :

>
>
> Mitch Gitman wrote:
>>
>> Stephen, thanks for verifying this.
>>
>
> No problem.
>
>
> Mitch Gitman wrote:
>>
>> ...for the new strict rule introduced with IvyDE 2.0.0 beta1 where  
>> the
>> source artifacts have to be declared in the ivy.xml to get picked up
>>
>
> I'm missing something--this rule makes sense to me. How did IvyDE  
> previously
> go about finding source artifacts if they were not declared in the  
> ivy.xml
> file?

It was trying some http requests based on the jar location, adding  
some -sources suffix. It was making the resolve quite slow due to some  
useless http connections, so it was removed as soon as sources were  
declared by Ivy when parsing pom.xml.

Nicolas

[Stephen Haberman]

Nicolas Lalevée wrote:

Sorry I didn't have time to reply to this interesting discussion.  
Actually I already have that discussion around the same issue in pom /  
ivy mapping there:
http://old.nabble.com/pom2ivy-and-transitive-source-retrieving-to25112985.html#a25112985

I "just" (shame on me) forgot to actually implement the solution we  
agreed on. I'll do soon.

Ah, no, shame on me for not finding an existing discussion /and/ bug about this from only a few months ago.

Thanks for pointing that thread out--though I'm disappointed Xavier didn't like jars+sources in the same conf. :-)

I suppose if its gets the job done, I'll be fine, but the transitive-sources conf seems even uglier than the existing sources conf. Granted, it does gain the requested behavior with backwards compatibility.

I wonder if there is an another thing lurking here--e.g. something like "primary" vs. "secondary" artifacts, where the primary artifacts are those you want downloaded with a "*" type pattern, but secondary artifacts need explicitly opted-in to a download, e.g. with a type=source (or whatever their type may be).

That being said, I usually prefer minimalist solutions (hence not liking the extra "sources" conf), and adding yet another attribute just for this one use case is not exactly minimalist.

But I like the conceptual integrity (to me anyway) of jars+sources being in the same conf that I suppose I'm reaching a little bit to see if it could swing that way.

(...ah, well, even if you set foo=secondary on the source artifacts, and put them in the master conf, they wouldn't get downloaded without a type filter for master conf, but then they also wouldn't get downloaded without a type filter for the sources conf. So my attempt at backwards compatibility failed.)

Thanks,
Stephen

[Mitch Gitman]

Question for Nicolas Lalevée. And Nicolas, please forgive me for putting you
on the spot. (Actually, there's nothing stopping anyone else from taking a
shot at this.)

Set aside the whole matter of translating a POM. Nicolas, I know you're an
advocate of the "source as type" approach in general as opposed to "source
as conf"--having written in the past:
"Rather than choosing the configuration to select or not the javadocs and
the sources, you should use the types. Configurations and types are
orthogonal concepts, you should try to not mix them."
http://old.nabble.com/Re%3A-IvyDE-automatically-include-javadoc-and-source-p25081870.html

What's your best argument as to why they *should be *considered orthogonal
concepts?

On Sun, Nov 1, 2009 at 7:14 AM, Stephen Haberman
<stephen@...>wrote:

>
>
>
> Nicolas Lalevée wrote:
> >
> > Sorry I didn't have time to reply to this interesting discussion.
> > Actually I already have that discussion around the same issue in pom /
> > ivy mapping there:
> >
> http://old.nabble.com/pom2ivy-and-transitive-source-retrieving-to25112985.html#a25112985
> >
> > I "just" (shame on me) forgot to actually implement the solution we
> > agreed on. I'll do soon.
> >
>
> Ah, no, shame on me for not finding an existing discussion /and/ bug about
> this from only a few months ago.
>
> Thanks for pointing that thread out--though I'm disappointed Xavier didn't
> like jars+sources in the same conf. :-)
>
> I suppose if its gets the job done, I'll be fine, but the
> transitive-sources
> conf seems even uglier than the existing sources conf. Granted, it does
> gain
> the requested behavior with backwards compatibility.
>
> I wonder if there is an another thing lurking here--e.g. something like
> "primary" vs. "secondary" artifacts, where the primary artifacts are those
> you want downloaded with a "*" type pattern, but secondary artifacts need
> explicitly opted-in to a download, e.g. with a type=source (or whatever
> their type may be).
>
> That being said, I usually prefer minimalist solutions (hence not liking
> the
> extra "sources" conf), and adding yet another attribute just for this one
> use case is not exactly minimalist.
>
> But I like the conceptual integrity (to me anyway) of jars+sources being in
> the same conf that I suppose I'm reaching a little bit to see if it could
> swing that way.
>
> (...ah, well, even if you set foo=secondary on the source artifacts, and
> put
> them in the master conf, they wouldn't get downloaded without a type filter
> for master conf, but then they also wouldn't get downloaded without a type
> filter for the sources conf. So my attempt at backwards compatibility
> failed.)
>
> Thanks,
> Stephen
>
>
>
>
> --
> View this message in context:
> http://old.nabble.com/sources-as-conf-or-type-tp26028446p26148902.html
> Sent from the ivy-user mailing list archive at Nabble.com.
>
>

[Nicolas Lalevée]

Le 2 nov. 2009 à 16:54, Mitch Gitman a écrit :

> Question for Nicolas Lalevée. And Nicolas, please forgive me for putting you
> on the spot. (Actually, there's nothing stopping anyone else from taking a
> shot at this.)
>
> Set aside the whole matter of translating a POM. Nicolas, I know you're an
> advocate of the "source as type" approach in general as opposed to "source
> as conf"--having written in the past:
> "Rather than choosing the configuration to select or not the javadocs and
> the sources, you should use the types. Configurations and types are
> orthogonal concepts, you should try to not mix them."
> http://old.nabble.com/Re%3A-IvyDE-automatically-include-javadoc-and-source-p25081870.html
>
> What's your best argument as to why they *should be *considered orthogonal
> concepts?

First, as the time I wrote it, I was thinking that this was the only good way of managing sources. I didn't realized that the pom were managed differently. Since the several discussions here I understand the other point of view.

Per se, types and configurations are two orthogonal concepts, as types and configurations are independent. Configurations names are not restricted by the types values, types values are not restricted by the configurations names. So the debate is how do you interpret them and how do you use them.

Configurations is "a way to use or construct a module" as the doc says [1] and I think everybody agrees. In the tutorial [2] you can see that we can declare different use of the module, different use that can be dependent to each other. So requiring the sources would fit this description, requiring the sources of a module can be seen as a way to use the module.

Then comes the need to declares sources (note that I don't talk yet about type or configuration for sources in this paragraph, just some abstract "requirement"). The source requirement is quite orthogonal to the "uses" illustrated in the tutorial: homemade-impl, cc-impl, api. If "api" is needed, "sources" is not necessary to use "api". And actually I don't know what it would really mean to ask for "source" without specifying the requirement for "homemade-impl", "cc-impl" or "api". This little point make me think that configurations are not per se designed for this.

So at some point we want to declare that an artifact is "api" and "source". And as a user of the module we want to declare that we want the "sources" of "homemade-impl". There is two ways:
* "source" as configuration
* "source" as type

If configuration is chosen, you need to:
* declare the configuration "source"
* declare that each source artifact is of the configuration "source"
* don't forget the transitivity in the dependencies (the current issue with the pom)
And the client needs to specify: conf="homemade-impl + source" (+ is configuration intersection since Ivy 2.1 [3])

If type is chosen, you need to:
* declare that each source artifact is of the type "source"
And the client needs to specify: conf="homemade-impl" type="source"

So both configuration and type can express sources. And the drawback about requiring type naming to be well known ("src", "source", "sources" and not others), I think this is the same with the name of the configurations (and actually more painful as transitivity is managed "manually").

Finally I just prefer type because it seems simpler to me. And pragmatically because Ivy didn't support configuration intersection before 2.1. And IvyDE today find sources thanks to the "type".

Nicolas

[1] http://ant.apache.org/ivy/history/latest-milestone/concept.html#configurations
[2] http://ant.apache.org/ivy/history/latest-milestone/tutorial/conf.html
[3] http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependency.html


PS: sorry for the delay in my reply but I find this topic too interesting to do a short reply ;)

>
> On Sun, Nov 1, 2009 at 7:14 AM, Stephen Haberman
> <stephen@...>wrote:
>
>>
>>
>>
>> Nicolas Lalevée wrote:
>>>
>>> Sorry I didn't have time to reply to this interesting discussion.
>>> Actually I already have that discussion around the same issue in pom /
>>> ivy mapping there:
>>>
>> http://old.nabble.com/pom2ivy-and-transitive-source-retrieving-to25112985.html#a25112985
>>>
>>> I "just" (shame on me) forgot to actually implement the solution we
>>> agreed on. I'll do soon.
>>>
>>
>> Ah, no, shame on me for not finding an existing discussion /and/ bug about
>> this from only a few months ago.
>>
>> Thanks for pointing that thread out--though I'm disappointed Xavier didn't
>> like jars+sources in the same conf. :-)
>>
>> I suppose if its gets the job done, I'll be fine, but the
>> transitive-sources
>> conf seems even uglier than the existing sources conf. Granted, it does
>> gain
>> the requested behavior with backwards compatibility.
>>
>> I wonder if there is an another thing lurking here--e.g. something like
>> "primary" vs. "secondary" artifacts, where the primary artifacts are those
>> you want downloaded with a "*" type pattern, but secondary artifacts need
>> explicitly opted-in to a download, e.g. with a type=source (or whatever
>> their type may be).
>>
>> That being said, I usually prefer minimalist solutions (hence not liking
>> the
>> extra "sources" conf), and adding yet another attribute just for this one
>> use case is not exactly minimalist.
>>
>> But I like the conceptual integrity (to me anyway) of jars+sources being in
>> the same conf that I suppose I'm reaching a little bit to see if it could
>> swing that way.
>>
>> (...ah, well, even if you set foo=secondary on the source artifacts, and
>> put
>> them in the master conf, they wouldn't get downloaded without a type filter
>> for master conf, but then they also wouldn't get downloaded without a type
>> filter for the sources conf. So my attempt at backwards compatibility
>> failed.)
>>
>> Thanks,
>> Stephen
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/sources-as-conf-or-type-tp26028446p26148902.html
>> Sent from the ivy-user mailing list archive at Nabble.com.
>>
>>

[Stephen Haberman]

Nicolas Lalevée wrote:

Finally I just prefer type because it seems simpler to me. And pragmatically because Ivy didn't support configuration intersection before 2.1. And IvyDE today find sources thanks to the "type".

Thanks for the reply, Nicolas.

I'm starting to deprecate the separate sources conf on projects I work on and thought to check one more thing--I haven't really started using the ivyroundup repo, but it seems promising.

Poking around, the projects in ivyroundup I looked at seemed to have source artifacts in the default conf and not have a separate sources conf.

Is this the official style of the ivyroundup repo or does it vary based on who setup each package?

I checked the style guide [1] and didn't see anything about sources in particular. Would it be worth making the convention explicit?

Thanks,
Stephen

[1]: http://code.google.com/p/ivyroundup/wiki/ModuleMaintainerGuidelines

[Archie Cobbs-3]

On Mon, Nov 23, 2009 at 9:13 AM, Stephen Haberman
<stephen@...>wrote:

> Poking around, the projects in ivyroundup I looked at seemed to have source
> artifacts in the default conf and not have a separate sources conf.
>
> Is this the official style of the ivyroundup repo or does it vary based on
> who setup each package?
>
> I checked the style guide [1] and didn't see anything about sources in
> particular. Would it be worth making the convention explicit?
>

Most modules in Ivy Roundup publish just a single source artifact (usually
called "sources.zip"), and it therefore is not assigned to a specific
configuration.

There are a handful of modules that publish multiple source artifacts, and
for the most part these do assign the source artifacts to the appropriate
configurations... for example,
Spring<file:///home/archie/projects/ivyroundup/trunk/repo/modules/org.springframework/spring/2.5.6/ivy.xml>and
Hibernate<file:///home/archie/projects/ivyroundup/trunk/repo/modules/org.hibernate/hibernate/3.3.2.GA/ivy.xml>
.

I will add explicit guidelines to the style guide to this effect.

Thanks,
-Archie

--
Archie L. Cobbs