Refactorability proposal...

[Bryan Atsatt]

(FYI: I've discussed this with Stanley but wanted to ensure it was
visible to everyone.)

Proposal

1. Eliminate runtime use of import-by-module in the JAM system:

    a. Support the (very convenient) import-by-module at the source level.
    b. jam tool transforms import-by-module to a list of
import-by-package statements.

2. Add APIs to fully support import-by-package in the JAM system:

   a. Support Version annotation in package-info.java (or in
module-info.java). If a
       package does not declare a version, it "inherits" that of the
enclosing module.
   b. Add ImportPackage annotation with version constraints.

3. Add APIs to fully support import-by-package in the abstract framework:
 
    a. Add methods to Query to produce import-by-package nodes.
    b. Replace Query.getIndexableNames() with fully generic variants (I
proposed
        a solution here previously which I will re-post).

Rationale

Module refactoring is inevitable, particularly during the transition
from the current, effectively flat class space to a fine-grained space
provided by module systems. We have significant experience with this
issue at Oracle (with the transition to our own module system), and OSGi
best-practices for conversion include starting with everything in one
bundle and then separating out pieces as experience is gained.

A very common pattern, in our experience, is for developers to start
with many extra jars in their initial module (a mini version of
class-path hell). As that module is put into wider use, someone
discovers that package X is also contained in their module, and that
duplication either leads to runtime conflicts (very bad), or just plain
footprint bloat. The obvious answer is to put package X in a separate
module, and have everyone share it via imports.

But... not so fast. If there are consumers of that module who import it
by module name alone, then pulling X out of it will cause those
importers to break. And if it is possible for your module to have been
imported by *anyone* by name alone, then you are stuck: either you break
them or you live with the incorrect granularity (which just isn't an
option in the conflict scenarios). Not a happy choice.

Originally, I had proposed to do away with import-by-module altogether,
both to avoid this problem and to eliminate the conceptual disconnect.
Your code does not today contain import statements that name *jars*, it
names packages and/or specific classes in those packages. Why invent a
new system that takes such a large step backwards?

The answer is simply convenience. Imagine a module that contains 100
packages and it is obvious that writing a single import statement is far
easier than discovering and explicitly writing all the package imports.
Yes, IDEs will likely mostly eliminate this issue, but it still makes
sense to be able to do this by hand.

This proposal is an attempt to maintain the convenience while adding the
crucial ability to safely refactor: step 1b is the central idea.

// Bryan

[Adrian Brock-4]

Refactoring:

Aren't there cases where resolving a module import
into its packages during deployment into the repository
will break with some refactorings?

I'm not talking about the more stable module refactorings
where modules are broken into other modules (which
you seem to be addressing).

This is more development time practices where somebody
is refactoring module contents. i.e. the importing
module could end up with a "stale" view of what it
should be importing.

A not very good example would be my module B imports a module A
where A exports com.acme.a.interfaces and
com.acme.a.implementation
I don't use com.acme.a.implementation explicitly so it
shouldn't really be one of my constraints.

Somebody then refactors module B such that the
implementation is now in com.acme.a.impl
but my module wants to import the now non-existant
(or at least stale) com.acme.a.implementation.

Overrides in the repository:

I know we avoid these discussions on the list since it
is really a tooling issue.

But it is a lot easier for a sysadmin to change
a single version constraint for a module import
than it is to have to figure out
what package versions they need to change on the
package imports and change each one individually.

Basically, the convenience of import-by-module
extends beyond the development/source code phase.

OSGi/291

There's still going to be a kind of module import
somewhere to handle OSGi's bundle import constraint.
i.e. OSGi bundle B issues a request to import bundle A
which is really a 277 module.

On Tue, 2008-06-10 at 18:27 -0700, Bryan Atsatt wrote:

> (FYI: I've discussed this with Stanley but wanted to ensure it was
> visible to everyone.)
>
> Proposal
>
> 1. Eliminate runtime use of import-by-module in the JAM system:
>
>     a. Support the (very convenient) import-by-module at the source level.
>     b. jam tool transforms import-by-module to a list of
> import-by-package statements.
>
> 2. Add APIs to fully support import-by-package in the JAM system:
>
>    a. Support Version annotation in package-info.java (or in
> module-info.java). If a
>        package does not declare a version, it "inherits" that of the
> enclosing module.
>    b. Add ImportPackage annotation with version constraints.
>
> 3. Add APIs to fully support import-by-package in the abstract framework:
>  
>     a. Add methods to Query to produce import-by-package nodes.
>     b. Replace Query.getIndexableNames() with fully generic variants (I
> proposed
>         a solution here previously which I will re-post).
>
> Rationale
>
> Module refactoring is inevitable, particularly during the transition
> from the current, effectively flat class space to a fine-grained space
> provided by module systems. We have significant experience with this
> issue at Oracle (with the transition to our own module system), and OSGi
> best-practices for conversion include starting with everything in one
> bundle and then separating out pieces as experience is gained.
>
> A very common pattern, in our experience, is for developers to start
> with many extra jars in their initial module (a mini version of
> class-path hell). As that module is put into wider use, someone
> discovers that package X is also contained in their module, and that
> duplication either leads to runtime conflicts (very bad), or just plain
> footprint bloat. The obvious answer is to put package X in a separate
> module, and have everyone share it via imports.
>
> But... not so fast. If there are consumers of that module who import it
> by module name alone, then pulling X out of it will cause those
> importers to break. And if it is possible for your module to have been
> imported by *anyone* by name alone, then you are stuck: either you break
> them or you live with the incorrect granularity (which just isn't an
> option in the conflict scenarios). Not a happy choice.
>
> Originally, I had proposed to do away with import-by-module altogether,
> both to avoid this problem and to eliminate the conceptual disconnect.
> Your code does not today contain import statements that name *jars*, it
> names packages and/or specific classes in those packages. Why invent a
> new system that takes such a large step backwards?
>
> The answer is simply convenience. Imagine a module that contains 100
> packages and it is obvious that writing a single import statement is far
> easier than discovering and explicitly writing all the package imports.
> Yes, IDEs will likely mostly eliminate this issue, but it still makes
> sense to be able to do this by hand.
>
> This proposal is an attempt to maintain the convenience while adding the
> crucial ability to safely refactor: step 1b is the central idea.
>
> // Bryan

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

[Adrian Brock-4]

Another issue would be how you handle the generated package
constraints.

e.g. Suppose you a module JavaEE:4.0.0 that
contains packages javax.jms:1.1.0, javax.resource:1.5.0, etc.

I then declare a module constraint
JavaEE:[4.0.0, 5.0.0)
i.e. I work with JavaEE4 but not JavaEE5

What do the package constraints resolve to for jms and jca
if we don't have a JavaEE5 module in the repository? :-)

JavaEE is not a very a good example since package versions
wouldn't change across a major release, but they could
in other cases or more general module version constraints.

On Tue, 2008-06-10 at 18:27 -0700, Bryan Atsatt wrote:

> (FYI: I've discussed this with Stanley but wanted to ensure it was
> visible to everyone.)
>
> Proposal
>
> 1. Eliminate runtime use of import-by-module in the JAM system:
>
>     a. Support the (very convenient) import-by-module at the source level.
>     b. jam tool transforms import-by-module to a list of
> import-by-package statements.
>
> 2. Add APIs to fully support import-by-package in the JAM system:
>
>    a. Support Version annotation in package-info.java (or in
> module-info.java). If a
>        package does not declare a version, it "inherits" that of the
> enclosing module.
>    b. Add ImportPackage annotation with version constraints.
>
> 3. Add APIs to fully support import-by-package in the abstract framework:
>  
>     a. Add methods to Query to produce import-by-package nodes.
>     b. Replace Query.getIndexableNames() with fully generic variants (I
> proposed
>         a solution here previously which I will re-post).
>
> Rationale
>
> Module refactoring is inevitable, particularly during the transition
> from the current, effectively flat class space to a fine-grained space
> provided by module systems. We have significant experience with this
> issue at Oracle (with the transition to our own module system), and OSGi
> best-practices for conversion include starting with everything in one
> bundle and then separating out pieces as experience is gained.
>
> A very common pattern, in our experience, is for developers to start
> with many extra jars in their initial module (a mini version of
> class-path hell). As that module is put into wider use, someone
> discovers that package X is also contained in their module, and that
> duplication either leads to runtime conflicts (very bad), or just plain
> footprint bloat. The obvious answer is to put package X in a separate
> module, and have everyone share it via imports.
>
> But... not so fast. If there are consumers of that module who import it
> by module name alone, then pulling X out of it will cause those
> importers to break. And if it is possible for your module to have been
> imported by *anyone* by name alone, then you are stuck: either you break
> them or you live with the incorrect granularity (which just isn't an
> option in the conflict scenarios). Not a happy choice.
>
> Originally, I had proposed to do away with import-by-module altogether,
> both to avoid this problem and to eliminate the conceptual disconnect.
> Your code does not today contain import statements that name *jars*, it
> names packages and/or specific classes in those packages. Why invent a
> new system that takes such a large step backwards?
>
> The answer is simply convenience. Imagine a module that contains 100
> packages and it is obvious that writing a single import statement is far
> easier than discovering and explicitly writing all the package imports.
> Yes, IDEs will likely mostly eliminate this issue, but it still makes
> sense to be able to do this by hand.
>
> This proposal is an attempt to maintain the convenience while adding the
> crucial ability to safely refactor: step 1b is the central idea.
>
> // Bryan

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

[Bryan Atsatt]

The proposal is to resolve module-->packages at *build* time, not
deploy. This way, the mapping is guaranteed to be valid.

And yes, this does make the sysadmin job a bit more difficult, but that
should be relatively easy to address with tooling.

The main point is that relying on import-by-module at runtime guarantees
that refactoring is problematic. My proposal seems like a reasonable way
to deal with this, but I wanted to start this discussion to explore
others as well.

Any thoughts on how to solve this problem?

// Bryan

Adrian Brock wrote:

> Refactoring:
>
> Aren't there cases where resolving a module import
> into its packages during deployment into the repository
> will break with some refactorings?
>
> I'm not talking about the more stable module refactorings
> where modules are broken into other modules (which
> you seem to be addressing).
>
> This is more development time practices where somebody
> is refactoring module contents. i.e. the importing
> module could end up with a "stale" view of what it
> should be importing.
>
> A not very good example would be my module B imports a module A
> where A exports com.acme.a.interfaces and
> com.acme.a.implementation
> I don't use com.acme.a.implementation explicitly so it
> shouldn't really be one of my constraints.
>
> Somebody then refactors module B such that the
> implementation is now in com.acme.a.impl
> but my module wants to import the now non-existant
> (or at least stale) com.acme.a.implementation.
>
> Overrides in the repository:
>
> I know we avoid these discussions on the list since it
> is really a tooling issue.
>
> But it is a lot easier for a sysadmin to change
> a single version constraint for a module import
> than it is to have to figure out
> what package versions they need to change on the
> package imports and change each one individually.
>
> Basically, the convenience of import-by-module
> extends beyond the development/source code phase.
>
> OSGi/291
>
> There's still going to be a kind of module import
> somewhere to handle OSGi's bundle import constraint.
> i.e. OSGi bundle B issues a request to import bundle A
> which is really a 277 module.
>
> On Tue, 2008-06-10 at 18:27 -0700, Bryan Atsatt wrote:
>  
>> (FYI: I've discussed this with Stanley but wanted to ensure it was
>> visible to everyone.)
>>
>> Proposal
>>
>> 1. Eliminate runtime use of import-by-module in the JAM system:
>>
>>     a. Support the (very convenient) import-by-module at the source level.
>>     b. jam tool transforms import-by-module to a list of
>> import-by-package statements.
>>
>> 2. Add APIs to fully support import-by-package in the JAM system:
>>
>>    a. Support Version annotation in package-info.java (or in
>> module-info.java). If a
>>        package does not declare a version, it "inherits" that of the
>> enclosing module.
>>    b. Add ImportPackage annotation with version constraints.
>>
>> 3. Add APIs to fully support import-by-package in the abstract framework:
>>  
>>     a. Add methods to Query to produce import-by-package nodes.
>>     b. Replace Query.getIndexableNames() with fully generic variants (I
>> proposed
>>         a solution here previously which I will re-post).
>>
>> Rationale
>>
>> Module refactoring is inevitable, particularly during the transition
>> from the current, effectively flat class space to a fine-grained space
>> provided by module systems. We have significant experience with this
>> issue at Oracle (with the transition to our own module system), and OSGi
>> best-practices for conversion include starting with everything in one
>> bundle and then separating out pieces as experience is gained.
>>
>> A very common pattern, in our experience, is for developers to start
>> with many extra jars in their initial module (a mini version of
>> class-path hell). As that module is put into wider use, someone
>> discovers that package X is also contained in their module, and that
>> duplication either leads to runtime conflicts (very bad), or just plain
>> footprint bloat. The obvious answer is to put package X in a separate
>> module, and have everyone share it via imports.
>>
>> But... not so fast. If there are consumers of that module who import it
>> by module name alone, then pulling X out of it will cause those
>> importers to break. And if it is possible for your module to have been
>> imported by *anyone* by name alone, then you are stuck: either you break
>> them or you live with the incorrect granularity (which just isn't an
>> option in the conflict scenarios). Not a happy choice.
>>
>> Originally, I had proposed to do away with import-by-module altogether,
>> both to avoid this problem and to eliminate the conceptual disconnect.
>> Your code does not today contain import statements that name *jars*, it
>> names packages and/or specific classes in those packages. Why invent a
>> new system that takes such a large step backwards?
>>
>> The answer is simply convenience. Imagine a module that contains 100
>> packages and it is obvious that writing a single import statement is far
>> easier than discovering and explicitly writing all the package imports.
>> Yes, IDEs will likely mostly eliminate this issue, but it still makes
>> sense to be able to do this by hand.
>>
>> This proposal is an attempt to maintain the convenience while adding the
>> crucial ability to safely refactor: step 1b is the central idea.
>>
>> // Bryan
>>    

[Stanley M. Ho]

Hi Bryan,

I agreed that refactorability is indeed an important issue we need to
address. That said, there are couple concerns I have related to step 1b.
I'm not arguing against import-by-package at runtime; only against the
transformation of import-by-module to import-by-package at compile-time.

A compile/build-time transformation from import-by-module to
import-by-package could fundamentally be done in one of two ways.

First, it could involve analysis of the module's method signatures. The
issue is that the analysis may not be accurate all the time (e.g. code
that uses reflection, Class.forName(), etc.). The programmer will then
be required do extra trial-and-error steps to fix up the dependencies in
the metadata, and this is not acceptable.

Second, it could simply expand import-by-module as the set of exported
packages in the imported module, then there is a problem when migrating
from one version of an imported module to a newer one and the set of
packages have changed. Suppose a core module in JDK 7 exports package P
and the one in JDK 8 exports packages P and Q. If a module imports JDK
7+ and is compiled against JDK 7, step 1b will transform the module to
import only the packages that are known in the core module in JDK 7.
When this module runs in JDK 8 or later, it will not see the new
packages. This is problematic because many modules will compile against
an older version of the JDK and want to call new API in newer JDK
through reflection; in general, this situation applies to using other
libraries as well. This will also cause problem in applications that
involve weaving at runtime and the exact required packages are not known
at compile-time.

There are some general problems with expanding modules to packages at
compile/build-time, regardless of how the expansion is achieved.

First, there are modules (e.g. classpath module) that cannot logically
have exported packages which are known at compile-time and at runtime,
thus it would be impossible to perform the transformation in this case.
(Technically, we can go through every JAR in the classpath in every
launch to find out the exports of the classpath module at runtime, but
this has huge startup performance impact.)

Second, it implies that people export packages but import modules. This
will be hard to explain to beginners.

Last, but perhaps the most important, it means that unresolveable
packages at runtime will generate error messages which the programmer
must manually map back to imported modules and the transformation
process implemented by a tool. The fundamental aim of a programming
system should be to reduce the semantic gap between what a programmer
sees in code and what a programmer sees at execution; the mapping from
packages back to modules increases the semantic gap.

I accept that pushing module imports through to runtime makes it hard to
"rewrite" a program (i.e. to refactor module members) but it makes it
easy to "read" the program (in the sense of a small semantic gap between
what imports are read/written and what imports are executed). A core
Java principle has always been that reading is more important than
writing, so we do not believe increased ease of refactoring on
relatively rare occasions is worth complicating every single reading of
a dependency.

I still think the refactorability problem you brought up is very
important to address. That said, the proposed transformation seems to
have many issues as an acceptable general solution.

- Stanley

[Bryan Atsatt]

Stanley M. Ho wrote:

> Hi Bryan,
>
> I agreed that refactorability is indeed an important issue we need to
> address. That said, there are couple concerns I have related to step
> 1b. I'm not arguing against import-by-package at runtime; only against
> the transformation of import-by-module to import-by-package at
> compile-time.
>
> A compile/build-time transformation from import-by-module to
> import-by-package could fundamentally be done in one of two ways.
>
> First, it could involve analysis of the module's method signatures.
> The issue is that the analysis may not be accurate all the time (e.g.
> code that uses reflection, Class.forName(), etc.). The programmer will
> then be required do extra trial-and-error steps to fix up the
> dependencies in the metadata, and this is not acceptable.
>
> Second, it could simply expand import-by-module as the set of exported
> packages in the imported module, then there is a problem when
> migrating from one version of an imported module to a newer one and
> the set of packages have changed. Suppose a core module in JDK 7
> exports package P and the one in JDK 8 exports packages P and Q. If a
> module imports JDK 7+ and is compiled against JDK 7, step 1b will
> transform the module to import only the packages that are known in the
> core module in JDK 7. When this module runs in JDK 8 or later, it will
> not see the new packages. This is problematic because many modules
> will compile against an older version of the JDK and want to call new
> API in newer JDK through reflection; in general, this situation
> applies to using other libraries as well. This will also cause problem
> in applications that involve weaving at runtime and the exact required
> packages are not known at compile-time.

This is why OSGi has dynamic-import. And we need it as well. The fact
that a forName/loadClass() happens to work *without* an explicit import
is nothing but a ticking time-bomb.

>
> There are some general problems with expanding modules to packages at
> compile/build-time, regardless of how the expansion is achieved.
>
> First, there are modules (e.g. classpath module) that cannot logically
> have exported packages which are known at compile-time and at runtime,
> thus it would be impossible to perform the transformation in this
> case. (Technically, we can go through every JAR in the classpath in
> every launch to find out the exports of the classpath module at
> runtime, but this has huge startup performance impact.)

This seems like a very special case to me. First, a "classpath" module
is, by definition, a very strange beast in that it is fully dynamic: any
importer can be broken at any time by a simple command-line change. I
can certainly understand a classpath module existing to support legacy
code, but I have a hard time imagining creating a new module that would
express a dependency on such a thing.

Even so, I concede that there are likely special cases where import by
"something other than package name" at runtime is useful.

Really, I just want to make it possible for a module developer to be in
a position to "know" that splitting one module into multiple modules
will not break existing importers. I'm quite open to other solutions, as
long as they are reasonable.
>
> Second, it implies that people export packages but import modules.
> This will be hard to explain to beginners.
So explain that you both export and import packages, but we provide
syntactic sugar for both for convenience. (I'd even by happy to
eliminate that convenience on the import, to simplify things further, as
you know.)

This is a silly argument IMO (sorry). In modern IDEs, when you type in
an unqualified class name, the system pops up and asks you which one you
mean. Without it, you have to (gasp) type a bunch of import statements
at the top of your file. Why didn't James make this easier by allowing
us to import a jar?
>
> Last, but perhaps the most important, it means that unresolveable
> packages at runtime will generate error messages which the programmer
> must manually map back to imported modules and the transformation
> process implemented by a tool. The fundamental aim of a programming
> system should be to reduce the semantic gap between what a programmer
> sees in code and what a programmer sees at execution; the mapping from
> packages back to modules increases the semantic gap.
I agree there is a semantic gap, but not the one you cite.

Either refactoring is supported, or it isn't. If it isn't, this
particular error should not occur. If it is supported, but an
import-by-module exists, it will be even worse than you suggest: there
won't be *any* failure at resolution time.

Say I import "module X", which contains package X and also happens to
contain packages Y and Z which I "silently" depend on. I compile and run
fine, but in some future X is split into X, Y and Z modules, and, for
some reason module Z is not installed. With my proposal, resolution for
my module will fail ("package Z not found") iff there is no installed
module containing Z.

Without my proposal (or some other similar solution), resolution will
succeed when it shouldn't, and a NoClassDefFoundError will occur at some
random point in execution naming a class in a package that was never
explicitly imported. How is that better?
>
> I accept that pushing module imports through to runtime makes it hard
> to "rewrite" a program (i.e. to refactor module members) but it makes
> it easy to "read" the program (in the sense of a small semantic gap
> between what imports are read/written and what imports are executed).
> A core Java principle has always been that reading is more important
> than writing, so we do not believe increased ease of refactoring on
> relatively rare occasions is worth complicating every single reading
> of a dependency.
Well there's a distinct difference between our perceptions: "relatively
rare occasions" strikes me as extremely misleading. Though I have
significant, direct, and exactly analogous experience that tells me the
frequency will be higher than you seem to expect, I'm perfectly happy to
concede that it may be lower. But unless you believe that it will be
zero, we need a tenable solution.
> I still think the refactorability problem you brought up is very
> important to address.
Well at least we can agree on that :^). Any ideas that are less ugly
than mine?

// Bryan

[Adrian Brock-4]

On Thu, 2008-06-12 at 17:13 -0700, Bryan Atsatt wrote:
> Really, I just want to make it possible for a module developer to be in
> a position to "know" that splitting one module into multiple modules
> will not break existing importers. I'm quite open to other solutions, as
> long as they are reasonable.

But can't that also be achieved with the old module re-exporting the
split out module for backwards compatibility purposes?
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

[Adrian Brock-4]

On Thu, 2008-06-12 at 10:53 -0700, Bryan Atsatt wrote:
> The proposal is to resolve module-->packages at *build* time, not
> deploy. This way, the mapping is guaranteed to be valid.

Whether it is done at build time or deploy time,
my criticisms still apply.

Its still based upon what was in the repository at that time
and may not reflect what is in the repository at runtime.

Generating the package version constraints from a module constraint
is non-trivial in general.

> And yes, this does make the sysadmin job a bit more difficult, but
> that should be relatively easy to address with tooling.

The old, "A tool will solve the complexity problem" opt out. ;-)
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Adrian Brock
Chief Scientist
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

[Bryan Atsatt]

Yeah, that was my original proposal (seems like years ago now :^), but
it means that you can never escape from your mistakes.

As a newbie, you create module X containing your code in package X. But
you have a dependency on package Y, so, following the traditional
keep-adding-to-the-classpath-till-it-works model, you throw in package
Y. But, argh, Y depends on Z, so you throw that in as well. Being a
newbie, you export everything to keep it simple.

Sometime later you realize it was a mistake to export Y and Z: your code
didn't actually depend on Z, nor did it expose Y. And the version of Y
that you want to use now is available in a shiny new module.

So you refactor to remove packages Y and Z and import Y instead.

But for compatibility, you are forced to also:

1. Import Z.
2. Re-export both Y and Z.

Ugly. Far worse though: it binds your importers both to the version of Y
that you depend on (but don't expose in your code), and the version of Z
that you don't depend on at all!

And all of this is a direct consequence of import-by-module. With
import-by-package, this issue just... vanishes.

// Bryan

Adrian Brock wrote:

> On Thu, 2008-06-12 at 17:13 -0700, Bryan Atsatt wrote:
>  
>> Really, I just want to make it possible for a module developer to be in
>> a position to "know" that splitting one module into multiple modules
>> will not break existing importers. I'm quite open to other solutions, as
>> long as they are reasonable.
>>    
>
> But can't that also be achieved with the old module re-exporting the
> split out module for backwards compatibility purposes?
>  

[Bryan Atsatt]

Yes, this could be an issue. If JavaEE:4.0.0 merely imports and
re-exports the specific packages, the imports can just be copied out. If
it actually contains them, and they themselves declare their own
versions, the jam tool could emit imports for those specific versions,
but it cannot generate ranges.

This brings us back to the "contract" idea: a named/versioned collection
of import constraints. The indirection afforded by this could certainly
solve this problem.

And we do have it, sort of: a module that contains no classes or
resources itself, only imports and re-exports. In the current model,
this is a bit of a hack because the module will actually be instantiated
with its own loader, which is a waste of cycles/resources.

If, OTOH, we special case such "contract" ("indirect"?) modules such
that they are not instantiated (or at least don't have a loader) but act
only as a bag of import constraints that can be "borrowed" by an
importer, that seems like a reasonable solution.

This could be simplified by doing the borrowing at build time, as in my
proposal, at the cost of some loss of flexibility.

// Bryan


Adrian Brock wrote:

> Another issue would be how you handle the generated package
> constraints.
>
> e.g. Suppose you a module JavaEE:4.0.0 that
> contains packages javax.jms:1.1.0, javax.resource:1.5.0, etc.
>
> I then declare a module constraint
> JavaEE:[4.0.0, 5.0.0)
> i.e. I work with JavaEE4 but not JavaEE5
>
> What do the package constraints resolve to for jms and jca
> if we don't have a JavaEE5 module in the repository? :-)
>
> JavaEE is not a very a good example since package versions
> wouldn't change across a major release, but they could
> in other cases or more general module version constraints.
>
> On Tue, 2008-06-10 at 18:27 -0700, Bryan Atsatt wrote:
>  
>> (FYI: I've discussed this with Stanley but wanted to ensure it was
>> visible to everyone.)
>>
>> Proposal
>>
>> 1. Eliminate runtime use of import-by-module in the JAM system:
>>
>>     a. Support the (very convenient) import-by-module at the source level.
>>     b. jam tool transforms import-by-module to a list of
>> import-by-package statements.
>>
>> 2. Add APIs to fully support import-by-package in the JAM system:
>>
>>    a. Support Version annotation in package-info.java (or in
>> module-info.java). If a
>>        package does not declare a version, it "inherits" that of the
>> enclosing module.
>>    b. Add ImportPackage annotation with version constraints.
>>
>> 3. Add APIs to fully support import-by-package in the abstract framework:
>>  
>>     a. Add methods to Query to produce import-by-package nodes.
>>     b. Replace Query.getIndexableNames() with fully generic variants (I
>> proposed
>>         a solution here previously which I will re-post).
>>
>> Rationale
>>
>> Module refactoring is inevitable, particularly during the transition
>> from the current, effectively flat class space to a fine-grained space
>> provided by module systems. We have significant experience with this
>> issue at Oracle (with the transition to our own module system), and OSGi
>> best-practices for conversion include starting with everything in one
>> bundle and then separating out pieces as experience is gained.
>>
>> A very common pattern, in our experience, is for developers to start
>> with many extra jars in their initial module (a mini version of
>> class-path hell). As that module is put into wider use, someone
>> discovers that package X is also contained in their module, and that
>> duplication either leads to runtime conflicts (very bad), or just plain
>> footprint bloat. The obvious answer is to put package X in a separate
>> module, and have everyone share it via imports.
>>
>> But... not so fast. If there are consumers of that module who import it
>> by module name alone, then pulling X out of it will cause those
>> importers to break. And if it is possible for your module to have been
>> imported by *anyone* by name alone, then you are stuck: either you break
>> them or you live with the incorrect granularity (which just isn't an
>> option in the conflict scenarios). Not a happy choice.
>>
>> Originally, I had proposed to do away with import-by-module altogether,
>> both to avoid this problem and to eliminate the conceptual disconnect.
>> Your code does not today contain import statements that name *jars*, it
>> names packages and/or specific classes in those packages. Why invent a
>> new system that takes such a large step backwards?
>>
>> The answer is simply convenience. Imagine a module that contains 100
>> packages and it is obvious that writing a single import statement is far
>> easier than discovering and explicitly writing all the package imports.
>> Yes, IDEs will likely mostly eliminate this issue, but it still makes
>> sense to be able to do this by hand.
>>
>> This proposal is an attempt to maintain the convenience while adding the
>> crucial ability to safely refactor: step 1b is the central idea.
>>
>> // Bryan
>>