Osgifing Tomcat

[hgomez]

>  That's maven's problem - I don't think there is any value in
>  continuing this discussion,
>  again - if you can support maven by adding a build/maven directory and
>  whatever files
>  inside - you have my +1, I'm all for making it easy to build - as long
>  as the tools are not
>  intrusive and don't force in their religion and arbitrary conventions.

Ok, I'll work on this direction, keeping the actual layout and adding
files / folders to mimic the modules structure

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[djencks]

On Apr 22, 2008, at 6:25 PM, Paul Benedict wrote:

> Is that enough so that web applications, either as a whole or in  
> partial,
> can be upgraded without stopping them? It's my understanding that if  
> web
> applications are loaded in an OSGi classloader, these kind of things  
> are
> possible.
>
I've heard various claims of this nature from osgi zealots, but when  
talking to apparent experts the only things resembling this they  
seemed to know about were grad student experiments that did not have  
production use as even a far-in-the-future goal.  Do you know of any  
actual examples of this kind of behavior that actually work under load?

thanks
david jencks

> Paul
>
> On Tue, Apr 22, 2008 at 7:24 PM, Filip Hanik - Dev Lists <devlists@...
> >
> wrote:
>
>> Henri Gomez wrote:
>>
>>> Hi to all,
>>>
>>> Did there is plans, ideas or interest around about OSGI-fing  
>>> Tomcat ?
>>>
>>>
>> I've put a note about this a while ago in tomcat/trunk/PROPOSALS.txt
>> my original plan was just to make sure all the MANIFEST.MF for each  
>> file
>> would have enough in it so that each JAR can be a OSGi bundle.
>>
>> after that, one can add on as much or as little as one wishes
>>
>> Filip
>>
>>> Regards
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@...
>>> For additional commands, e-mail: dev-help@...
>>>
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@...
>> For additional commands, e-mail: dev-help@...
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Costin Manolache-2]

I'm not an expert, but I think I can tell you that yes, "hello world"
applications can be upgraded without stopping, real
applications can't.

As long as you use sessions or statics or you make config changes -
you have to restart the webapp.
OSGI is good of having 2 versions of a bundle running at the same time
- but it won't help you much, the servlet
engine needs to know where to send the requests, it has no clue a
request should go to the old version or the new.

Costin

On Thu, Apr 24, 2008 at 5:17 PM, David Jencks <david_jencks@...> wrote:

>
>  On Apr 22, 2008, at 6:25 PM, Paul Benedict wrote:
>
>
> > Is that enough so that web applications, either as a whole or in partial,
> > can be upgraded without stopping them? It's my understanding that if web
> > applications are loaded in an OSGi classloader, these kind of things are
> > possible.
> >
> >
>  I've heard various claims of this nature from osgi zealots, but when
> talking to apparent experts the only things resembling this they seemed to
> know about were grad student experiments that did not have production use as
> even a far-in-the-future goal.  Do you know of any actual examples of this
> kind of behavior that actually work under load?
>
>  thanks
>  david jencks
>
>
>
>
>
> > Paul
> >
> > On Tue, Apr 22, 2008 at 7:24 PM, Filip Hanik - Dev Lists
> <devlists@...>
> > wrote:
> >
> >
> > > Henri Gomez wrote:
> > >
> > >
> > > > Hi to all,
> > > >
> > > > Did there is plans, ideas or interest around about OSGI-fing Tomcat ?
> > > >
> > > >
> > > >
> > > I've put a note about this a while ago in tomcat/trunk/PROPOSALS.txt
> > > my original plan was just to make sure all the MANIFEST.MF for each file
> > > would have enough in it so that each JAR can be a OSGi bundle.
> > >
> > > after that, one can add on as much or as little as one wishes
> > >
> > > Filip
> > >
> > >
> > > > Regards
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@...
> > > > For additional commands, e-mail: dev-help@...
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@...
> > > For additional commands, e-mail: dev-help@...
> > >
> > >
> > >
> >
>
>
>  ---------------------------------------------------------------------
>  To unsubscribe, e-mail: dev-unsubscribe@...
>  For additional commands, e-mail: dev-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[hgomez]

>  OSGI is good of having 2 versions of a bundle running at the same time
>  - but it won't help you much, the servlet
>  engine needs to know where to send the requests, it has no clue a
>  request should go to the old version or the new.

May be additions to servlet specs should be planned, ie, by allowing
such informations about which version to take care of ?

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[hgomez]

>  I've heard various claims of this nature from osgi zealots, but when
> talking to apparent experts the only things resembling this they seemed to
> know about were grad student experiments that did not have production use as
> even a far-in-the-future goal.  Do you know of any actual examples of this
> kind of behavior that actually work under load?

We'll see how OSGI works underload with Glassfish v3.

BTW, performance is only a point in an IT infrastructure, ease of
deployment and administration is often more important (from my day to
day experience)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[djencks]

On Apr 24, 2008, at 11:11 PM, Henri Gomez wrote:

>> I've heard various claims of this nature from osgi zealots, but when
>> talking to apparent experts the only things resembling this they  
>> seemed to
>> know about were grad student experiments that did not have  
>> production use as
>> even a far-in-the-future goal.  Do you know of any actual examples  
>> of this
>> kind of behavior that actually work under load?
>
> We'll see how OSGI works underload with Glassfish v3.

Are they planning to support "gapless" redeployment of web apps using  
only osgi features, with no other servlet container support?  If so is  
can you point to an explanation of how they plan to do this?
>
>
> BTW, performance is only a point in an IT infrastructure, ease of
> deployment and administration is often more important (from my day to
> day experience)

I think its entirely possible for a servlet container to support  
gapless redeployment of web apps.  I don't see how this can be  
supplied or even helped by osgi itself.  As far as I can see, osgi has  
a fairly nice classloader that unfortunately requires all jars to be  
osgi-ified (a process that will take years and, judging by the  
semantically similar info in maven poms, mostly result in wrong  
information) and a fairly limited service and service dependency  
model, plus a lot of hype.  For instance I've seen this claim several  
times that osgi will by some magic make hot updates of production  
systems just work, but no one seems to be  able to tell me how its  
supposed to work.  The only actual info I've heard was about a student  
experiment that clearly would not work in a high-load environment.  
Personally I don't think this kind of feature will work as a component  
container feature without transactional memory; the only place I know  
of this being available is in haskell.


just my highly biased $0.02

david jencks

>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[jan.grant]

On Fri, 25 Apr 2008, David Jencks wrote:

> Are they planning to support "gapless" redeployment of web apps using only
> osgi features, with no other servlet container support?  If so is can you
> point to an explanation of how they plan to do this?

Since this requires state mutation (see the hooks that an Erlang OTP
application will supply for state upgrade, for instance) there won't be
a "no other hooks" version, I think :-)

This is doable (I've done it, but only as a proof of concept - as
pointed out, this is a long way away from a robust real-world approach)
but typically the applications we're redeploying for real here can (a)
stand a little downtime, or (b) can use staggered upgrades across a
cluster of app servers, (c) often come hand-in-hand with out-of-app DB
schema changes, and (d) consequently don't merit the huge effort
required to make this work.



--
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel [phone]   http://ioctl.org/jan/
Prolog in JavaScript: http://ioctl.org/logic/prolog-latest

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[hgomez]

>  Since this requires state mutation (see the hooks that an Erlang OTP
>  application will supply for state upgrade, for instance) there won't be
>  a "no other hooks" version, I think :-)
>
>  This is doable (I've done it, but only as a proof of concept - as
>  pointed out, this is a long way away from a robust real-world approach)
>  but typically the applications we're redeploying for real here can (a)
>  stand a little downtime, or (b) can use staggered upgrades across a
>  cluster of app servers, (c) often come hand-in-hand with out-of-app DB
>  schema changes, and (d) consequently don't merit the huge effort
>  required to make this work.

Good to see that we could discuss a little bit more than just OSGI/Maven.

Thanks

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Damien Bonvillain]

David Jencks wrote:
 > On Apr 24, 2008, at 11:11 PM, Henri Gomez wrote:
 > > We'll see how OSGI works underload with Glassfish v3.
> Are they planning to support "gapless" redeployment of web apps using
> only osgi features, with no other servlet container support?  If so is
> can you point to an explanation of how they plan to do this?

Just my 2c: Eclipse, OSGified since 3.2, still doesn't fully support
updates without a JVM restart; and they have full control about the
whole aspects.

A vocabulary point though, we shouldn't talk about OSGification, but
about jsrtwonineonification.

Damien

PS: usually only reader on this ML

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Costin Manolache-2]

On Fri, Apr 25, 2008 at 12:49 AM, David Jencks <david_jencks@...> wrote:

>
>  On Apr 24, 2008, at 11:11 PM, Henri Gomez wrote:
>
>
> >
> > > I've heard various claims of this nature from osgi zealots, but when
> > > talking to apparent experts the only things resembling this they seemed
> to
> > > know about were grad student experiments that did not have production
> use as
> > > even a far-in-the-future goal.  Do you know of any actual examples of
> this
> > > kind of behavior that actually work under load?
> > >
> >
> > We'll see how OSGI works underload with Glassfish v3.
> >
>
>  Are they planning to support "gapless" redeployment of web apps using only
> osgi features, with no other servlet container support?  If so is can you
> point to an explanation of how they plan to do this?

There are many problems to solve for 'gapless' redeployment, but I
think the biggest is
sessions and statics. The classloader ( OSGI or not ) can keep
multiple versions of
the application loaded - but you need the mapper to know where to send
each request.
In theory it could be done even with the current tomcat, by including
the version in the cookie and changing the mapper to use it.

But I think the main question is if it's worth it - there is a lot of
complexity and many things that can go wrong. Very few advanced users
will be able to use this - in particular if they have production
environments and some release testing is required ( i.e. it would be
quite hard to automate and test what will happen in gapless case). The
alternative - having multiple machines/VMs running different versions
behind a load balancer - is quite safe and I suspect the 'advanced
users' who may need gapless would be better off using the old way.
I mean - if you run your app out of a single VM, probably you won't
need gapless, and if you already have load balancing/failover/multiple
servers - you have a better/safer solution.

Costin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[hgomez]

>  But I think the main question is if it's worth it - there is a lot of
>  complexity and many things that can go wrong. Very few advanced users
>  will be able to use this - in particular if they have production
>  environments and some release testing is required ( i.e. it would be
>  quite hard to automate and test what will happen in gapless case). The
>  alternative - having multiple machines/VMs running different versions
>  behind a load balancer - is quite safe and I suspect the 'advanced
>  users' who may need gapless would be better off using the old way.
>  I mean - if you run your app out of a single VM, probably you won't
>  need gapless, and if you already have load balancing/failover/multiple
>  servers - you have a better/safer solution.

Right.

BTW, I see OSGI for another purpose, creating a more modular Tomcat.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[pkriens]

> I've heard various claims of this nature from osgi zealots, but when  
> talking to apparent experts the only things resembling this they  
> seemed to know about were grad student experiments that did not have  
> production use as even a far-in-the-future goal.  Do you know of any  
> actual examples of this kind of behavior that actually work under load?
As one of the OSGi "zealots" I can inform you that currently BEA micro service architecture, IBM Websphere 6.1, Jonas are fully osgified and used in production. Also Spring is now osgifing all their code and seeing it as a strategic advantage. JBoss is working hard on being compatible with OSGi. And of course, as was noted, there is now Glassfish. Add to this Eclipse's broad success with the IDE, RCP, eRCP, and now Eclipse on the server side. I am also aware that Geronimo was once ported to OSGi as a trial inside IBM.

I guess there likely have been some grad students been involved along the way.

Though the dynamics work quite well when properly designed for it, there are still many companies that use OSGi for its modularity enforcement. There is no policy that states that you must use dynamic loading in a production environment. Many companies use the dynamic facilities during during development but have strict controlled procedures for production, which of course works fine with OSGi based systems as well. However, over time people learn to trust the dynamics because they work quite well and reliable when properly used. However, modularizing your code has so many other advantages.

The dynamics were not so much a design goal as well a side effect of the strict enforcements of modularity. By minimizing the coupling between modules with private code and coupling through well defined services we found that you can actually link and unlink modules quite well when properly used. However, there are so many other advantages to strong modularization that this is just the icing of the cake.

Kind regards,

   Peter Kriens

djencks wrote:

On Apr 22, 2008, at 6:25 PM, Paul Benedict wrote:

> Is that enough so that web applications, either as a whole or in  
> partial,
> can be upgraded without stopping them? It's my understanding that if  
> web
> applications are loaded in an OSGi classloader, these kind of things  
> are
> possible.
>
I've heard various claims of this nature from osgi zealots, but when  
talking to apparent experts the only things resembling this they  
seemed to know about were grad student experiments that did not have  
production use as even a far-in-the-future goal.  Do you know of any  
actual examples of this kind of behavior that actually work under load?

thanks
david jencks


> Paul
>
> On Tue, Apr 22, 2008 at 7:24 PM, Filip Hanik - Dev Lists <[e-mail]
> >
> wrote:
>
>> Henri Gomez wrote:
>>
>>> Hi to all,
>>>
>>> Did there is plans, ideas or interest around about OSGI-fing  
>>> Tomcat ?
>>>
>>>
>> I've put a note about this a while ago in tomcat/trunk/PROPOSALS.txt
>> my original plan was just to make sure all the MANIFEST.MF for each  
>> file
>> would have enough in it so that each JAR can be a OSGi bundle.
>>
>> after that, one can add on as much or as little as one wishes
>>
>> Filip
>>
>>> Regards
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [e-mail]
>>> For additional commands, e-mail: [e-mail]
>>>
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [e-mail]
>> For additional commands, e-mail: [e-mail]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[pkriens]

The way the dynamics work in OSGi is only partly classloading. In the HTTP examples, we have defined an HttpService where you register a servlet. When a bundle (OSGi JAR) is started, it registers a servlet with the http service. When it is stopped, the Http Service detects this and removes the mapping. When you update the bundle, the bundle is first stopped, automatically cleaning up the servlet registration. Then the code is replaced though the old code remains in memory because other bundles could still import packages from this bundle. This remains until the packages are "refreshed". This will create a dependency graph of all affected bundles, stop these and start them again with a new class loader.

So there is no "magic". The key trick is that bundles have their own classloader and we can start and activate them as well as stop them at any moment in time. Obviously this works only if you make your bundles proper modules with loose coupling. If classes are shared between bundles willy nilly refreshing the packages will be a full restart.

Neither is it of course completely gapless. However, many, many people have installations where they replace their bundles with servlets on the fly because the Http server rarely if ever has a dependency on the application code.

Kind regards,

   Peter Kriens

Costin Manolache-2 wrote:

On Fri, Apr 25, 2008 at 12:49 AM, David Jencks <[e-mail]> wrote:
>
>  On Apr 24, 2008, at 11:11 PM, Henri Gomez wrote:
>
>
> >
> > > I've heard various claims of this nature from osgi zealots, but when
> > > talking to apparent experts the only things resembling this they seemed
> to
> > > know about were grad student experiments that did not have production
> use as
> > > even a far-in-the-future goal.  Do you know of any actual examples of
> this
> > > kind of behavior that actually work under load?
> > >
> >
> > We'll see how OSGI works underload with Glassfish v3.
> >
>
>  Are they planning to support "gapless" redeployment of web apps using only
> osgi features, with no other servlet container support?  If so is can you
> point to an explanation of how they plan to do this?

There are many problems to solve for 'gapless' redeployment, but I
think the biggest is
sessions and statics. The classloader ( OSGI or not ) can keep
multiple versions of
the application loaded - but you need the mapper to know where to send
each request.
In theory it could be done even with the current tomcat, by including
the version in the cookie and changing the mapper to use it.

But I think the main question is if it's worth it - there is a lot of
complexity and many things that can go wrong. Very few advanced users
will be able to use this - in particular if they have production
environments and some release testing is required ( i.e. it would be
quite hard to automate and test what will happen in gapless case). The
alternative - having multiple machines/VMs running different versions
behind a load balancer - is quite safe and I suspect the 'advanced
users' who may need gapless would be better off using the old way.
I mean - if you run your app out of a single VM, probably you won't
need gapless, and if you already have load balancing/failover/multiple
servers - you have a better/safer solution.

Costin

---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[pkriens]

> Just my 2c: Eclipse, OSGified since 3.2, still doesn't fully support
> updates without a JVM restart; and they have full control about the
> whole aspects.
Unfortunately there are tens of thousands of plugin writers out there that are not aware of the dynamics. Eclipse migrated to OSGi 4 years ago and did some incredible open heart surgery on their runtime engine to support OSGi. It was quite impressive how they replace their engine with an OSGi based version while existing plugins kept working. However, none of the existing code was ready for the consequences of dynamics. Plugins could add entries to menu's but this information came from a static file in the plugin, so there was no facility to remove this information.

Over time, more and more aspects will become dynamic but due to the nature of massively distributed development it will take some time before the code is fully dynamic aware. The extension registry is already an example where the extension points have become dynamic. Now only the apps are left ...

Kind regards,

    Peter Kriens

Damien Bonvillain wrote:

David Jencks wrote:
 > On Apr 24, 2008, at 11:11 PM, Henri Gomez wrote:
 > > We'll see how OSGI works underload with Glassfish v3.
> Are they planning to support "gapless" redeployment of web apps using
> only osgi features, with no other servlet container support?  If so is
> can you point to an explanation of how they plan to do this?

Just my 2c: Eclipse, OSGified since 3.2, still doesn't fully support
updates without a JVM restart; and they have full control about the
whole aspects.

A vocabulary point though, we shouldn't talk about OSGification, but
about jsrtwonineonification.

Damien

PS: usually only reader on this ML

---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[pkriens]

> I think the first main problem with these frameworks is how intrusive
> they are, esp compared with whet they provide :( I suppose there's no
> problem with doing a tomcat-osgi in the sandbox if people want to. As I
> said I really don't care.
We tried awfully hard to be not intrusive :-(

Actually, any JAR is an OSGi bundle by definition. Even if you do not add any extra information you will load and in certain cases, can be useful. However, it is really hard to provide functionality without providing any API ... By providing OSGi specific manifest headers you will make your code run in OSGi platforms unchanged. Today, many people are duplicating the effort to add these manifest headers for their products (Apache Felix commons, Eclipse Orbit, and hundreds of companies). Because this is not their core business they tend to use simple rules to modularize these libraries which works, but is not optimal because they are not intricately aware of the inter-package dependencies as you are. This kind of knowledge is required to properly modularize a complex system like Tomcat. When analyzing dependencies, one also usually finds reason to break the bigger chunks in smaller parts to remove many dependencies. Obviously, this is infeasible for outsiders to do right and even if they could, they would have to repeat this work for every release.

OSGi imports and exports are package based. That is why it is important to separate API and implementation. The manifest headers allow you to shield the implementation code from outside usage and only expose API packages. There is no need to put these packages in separate bundles, on the contrary it is usually very advantageous to combine them. Even if you have multiple bundles that implement the same API, there is no issue with exporting those API packages multiple times. The OSGi resolvers can handle that case and even handle the case when versions differ.

In the spirit of modularity, the OSGi API has only one mandatory part: the framework API. ALL other aspects are services (objects registered and unregistered dynamically in a service registry) and by their nature can be there or not. The framework API is quite small, it is one package with 26 classes/interfaces. This API is for management applications (install/start/stop/update/uninstall, introspection, etc) and for the use of the service registry. This low level API is often abstracted with "programming models" like Spring DM, Apache iPOJO, Declarative Services, SAT, etc. These models actually interact because they use the same underlying (dynamic) service model while not making the application handle the overhead of the service model.

In an ideal situation (well for me and other OSGi users), Tomcat would provide an Http Service when running on an OSGi  platform. However, just providing the right manifest headers would allow others to provide this service because it is not much code and I know many people have already done this with Tomcat.

Kind regards,

   Peter Kriens

remm wrote:

On Wed, 2008-04-23 at 16:00 +0200, Florent.BENOIT wrote:
> Also, for OSGi, as all is done by package (import/export) the first step
> is to be sure that API and Implementation are never in the same package
> name. So we can export APIs and keep private the implementation.

I think the first main problem with these frameworks is how intrusive
they are, esp compared with whet they provide :( I suppose there's no
problem with doing a tomcat-osgi in the sandbox if people want to. As I
said I really don't care.

Rémy



---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[pkriens]

> my feeling is though, is that you are going for the "mavenization" just
> to run the BND or BNE or whatever the plugin is called, that generates
> the OSGi manifests.
The project is called bnd (pronounce b and d). The jar can be used as an ant task, command utility, eclipse plugin, and library. The Apache Felix project (an OSGi implementation) has wrapped this library in plugin.

> having personally done that, I can say that it simply doesn't work. IT
> works in most cases, but not in Tomcat, and even in the cases it works,
> the output it produces into the manifest file is completely useless to
> the human eye. the amount of stuff it exports and imports is insane, in
> in most cases irrelevant to the data you actually wanna export/import
As the author of bnd I'd like to object :-) I assume you have used the wrap
function that just exports anything and imports everything optional.
This is a convenience function that is used to get
an existing JAR osgified quickly. Extremely useful if you need it, but it
obviously bnd can not guess your module boundaries. That is why it is
so important that the developers of a library do this because not only
do they know the boundaries, they can also improve them.

Therefore, bnd normally uses a recipe to steer it. This recipe allows you
to use wildcards and  variables to minimize the amount of manifest work. I.e.

   Import-Package: *

Will import everything (the default). However, if you have an ant task in your code but that task is only used in build time, then you like want to exclude the ant imports from your bundle to not create a dependency on it:

   Import-Package: !org.apache.ant.*, *

Inside the OSGi we use org.osgi.service.* packages for public API, and org.osgi.impl.* packages for private code. The private and export package headers allow wildcard packagenames. These packages are copied from the classpath into the JAR. This is included in the JAR like:

  Export-Package: org.osgi.service.log.*
  Private-Package: org.osgi.impl.log.*

So what bnd does is a combination of packaging packages from the classpath, analyzing the result, calculating the manifest, and then at the end a thorough analysis of all the headers so you catch errors before you deploy.

> that's why I posted my combined version of the Export/Import, nice and
> clean, when/if we do it on a per jar basis, I would hope we aim to
> produce the same quality data as the example I showed.
The problem is getting it right the first time and then maintaining
it over time. This kind of information is extremely
sensitive to refactoring, additions of packages, and use of
other code. The manifest is NOT intended for the human, it is
intended for an OSGi framework. I.e. the Java manifest rules state
that you must break after 72 characters, the uses statement is crucial
to maintain classpace consistency but is quite verbose. However, when
you use bnd, you can use the recipe file quite readable due the wildcards
and defaults. This recipe is supported in the ant task, eclipse plugin,
and command line. In the maven plugin it is encoded in the POM (and
can take variables from the pom).

Bnd can be used from the command line to show the information
in a much more readable form. Anyway, if bnd generates a lot of information
in the manifest, than  this information is really in your JAR. It creates
Import-Package statements  (and uses directives) based on an
analysis of your class files.

Some people do not trust bnd to generate the JAR. Bnd could just generate
the manifest if this is an issue. However, I would suggest to try to let bnd do
the packaging because it is quite flexible and reliable. One of the cool things (at least
I think it is) is that you can really compose your jars, regardless of project boundaries.

If there is anything special that you need from bnd, let me know.

Kind regards,

   Peter Kriens

Filip Hanik - Dev Lists wrote:

Henri Gomez wrote:
>>  I'm not sure it's the best idea, my goal is to move it out of sandbox,
>>  it already has enough experiments
>>  that need completion. and the main goal is to be 'lite' :-). It has a
>>  simple Addon mechanism, and I don't mind
>>  having an optional addon manager impl using OSGI - but I don't want to
>>  get distracted, I started
>>   tomcat-lite  experiment >2 years ago.
>>
>>    
>
> Yep, time to stabilize
>
>  
>>  The first part ( adding manifests to existing tomcat ) seems to have
>>  support so could be done in the trunk.
>>    
>
> And no consequences outside OSGI world
>
>  
>>  I don't see any reason for having non-intrusive OSGI support developed
>>  in sandbox - as long
>>  as the code is in a package that is excluded from the official distro,
>>  and is released as a separate
>>  unit it could live in trunk.  It'll need the 3 +1 to be released, of
>>  course - but the whole point of
>>   modularity is to be able to develop modules independent of the container.
>>    
>
> Right
>
>  
>>  IMO sandbox is for experiments that would replace existing tomcat
>>  components or core stuff. If you do it in
>>  trunk - it's easier to get it released eventually, and you may get
>>  better reviews and attention.
>>    
>
> Indeed
>
> I'll try to spend some time on mavenize tomcatlight first and how it
> could be done then for tomcat trunk.
>  
LOL, ouch, you just opened up the can of worms we thought we put a lid
on :) he he
basically, Tomcat 6 is mavenized, and we publish the individual JARs to
ASF Maven repo.
> Next how to OSGIfy the mavenized tomcats, experiences and advices welcomed here
>  
my feeling is though, is that you are going for the "mavenization" just
to run the BND or BNE or whatever the plugin is called, that generates
the OSGi manifests.
having personally done that, I can say that it simply doesn't work. IT
works in most cases, but not in Tomcat, and even in the cases it works,
the output it produces into the manifest file is completely useless to
the human eye. the amount of stuff it exports and imports is insane, in
in most cases irrelevant to the data you actually wanna export/import

that's why I posted my combined version of the Export/Import, nice and
clean, when/if we do it on a per jar basis, I would hope we aim to
produce the same quality data as the example I showed.

Filip
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [e-mail]
> For additional commands, e-mail: [e-mail]
>
>
>
>  


---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[pkriens]

> Someone recently pointed out to me that the Bnd tool comes with ant
> tasks. I haven't tried that (we've used maven in commons) and I
> suspect that there isn't the option to just produce the manifest
> (rather than jar and manifest) as there is in the maven plugin. If
> that was required then in might be worth requesting that:
If you need this, let me know. However, it might be an idea to start with a
separate set of jars that are produced by bnd, independently of the main jars.
You can setup bnd as ant task, give it a classpath of the primary jars and then
provide the recipe to create the manifest. This way, nobody has to become
nervous for touching the crown jewels.

> Are you talking about all the "uses" statements it adds? If so an
> option to turn off producing them was recently added to the Bnd tool -
> which improves the situation. It still wraps everything to 72
> characters which is harder to read than a manually created manifest -
> but I think automating this to reduce the chance of errors from
> manually keying in is worth the trade off.
The manifest is a machine readable file, it is not intended for humans. I.e. the 72
caracter limit is in the Java spec, not the OSGi spec. The bnd recipe file (which is a
properties file) is the one that is intended to be readable, not the generated manifest.

And yes, you can switch off the generation of uses but the uses directive has a very
important purpose: class space consistency. It ensures that the class loaders
will not put two classes in the same class space if they could clash. I.e. if both
classes use class com.foo.A, then the OSGi framework must ensure that com.foo.A
really comes from the same classloader for both bundles. The uses define for each
package, what other imported or exported packages are used. I -strongly-
recommend against switching this off. The option was only added for a very specific use case.

Bnd used as command line can also display this information in a less human unfriendly form.
I.e. you can use bnd as a dependency analyzer.

Kind regards,

  Peter Kriens

Niall Pemberton-2 wrote:

On Wed, Apr 23, 2008 at 6:56 PM, Filip Hanik - Dev Lists
<[e-mail]> wrote:
>
> Henri Gomez wrote:
>
> >
> > >  I'm not sure it's the best idea, my goal is to move it out of sandbox,
> > >  it already has enough experiments
> > >  that need completion. and the main goal is to be 'lite' :-). It has a
> > >  simple Addon mechanism, and I don't mind
> > >  having an optional addon manager impl using OSGI - but I don't want to
> > >  get distracted, I started
> > >  tomcat-lite  experiment >2 years ago.
> > >
> > >
> > >
> >
> > Yep, time to stabilize
> >
> >
> >
> > >  The first part ( adding manifests to existing tomcat ) seems to have
> > >  support so could be done in the trunk.
> > >
> > >
> >
> > And no consequences outside OSGI world
> >
> >
> >
> > >  I don't see any reason for having non-intrusive OSGI support developed
> > >  in sandbox - as long
> > >  as the code is in a package that is excluded from the official distro,
> > >  and is released as a separate
> > >  unit it could live in trunk.  It'll need the 3 +1 to be released, of
> > >  course - but the whole point of
> > >  modularity is to be able to develop modules independent of the
> container.
> > >
> > >
> >
> > Right
> >
> >
> >
> > >  IMO sandbox is for experiments that would replace existing tomcat
> > >  components or core stuff. If you do it in
> > >  trunk - it's easier to get it released eventually, and you may get
> > >  better reviews and attention.
> > >
> > >
> >
> > Indeed
> >
> > I'll try to spend some time on mavenize tomcatlight first and how it
> > could be done then for tomcat trunk.
> >
> >
>  LOL, ouch, you just opened up the can of worms we thought we put a lid on
> :) he he
>  basically, Tomcat 6 is mavenized, and we publish the individual JARs to ASF
> Maven repo.
>
>
> > Next how to OSGIfy the mavenized tomcats, experiences and advices welcomed
> here
> >
> >
>  my feeling is though, is that you are going for the "mavenization" just to
> run the BND or BNE or whatever the plugin is called, that generates the OSGi
> manifests.

Someone recently pointed out to me that the Bnd tool comes with ant
tasks. I haven't tried that (we've used maven in commons) and I
suspect that there isn't the option to just produce the manifest
(rather than jar and manifest) as there is in the maven plugin. If
that was required then in might be worth requesting that:

http://www.aqute.biz/Code/Bnd

>  having personally done that, I can say that it simply doesn't work. IT
> works in most cases, but not in Tomcat, and even in the cases it works, the
> output it produces into the manifest file is completely useless to the human
> eye. the amount of stuff it exports and imports is insane, in in most cases
> irrelevant to the data you actually wanna export/import

Are you talking about all the "uses" statements it adds? If so an
option to turn off producing them was recently added to the Bnd tool -
which improves the situation. It still wraps everything to 72
characters which is harder to read than a manually created manifest -
but I think automating this to reduce the chance of errors from
manually keying in is worth the trade off.

Niall

>  that's why I posted my combined version of the Export/Import, nice and
> clean, when/if we do it on a per jar basis, I would hope we aim to produce
> the same quality data as the example I showed.
>
>  Filip
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[Damien Bonvillain]

pkriens wrote:

>> Just my 2c: Eclipse, OSGified since 3.2, still doesn't fully support
>> updates without a JVM restart; and they have full control about the
>> whole aspects.
> Unfortunately there are tens of thousands of plugin writers out there that
> are not aware of the dynamics. Eclipse migrated to OSGi 4 years ago and did
> some incredible open heart surgery on their runtime engine to support OSGi.
> It was quite impressive how they replace their engine with an OSGi based
> version while existing plugins kept working. However, none of the existing
> code was ready for the consequences of dynamics. Plugins could add entries
> to menu's but this information came from a static file in the plugin, so
> there was no facility to remove this information.

To me, a webapp adds "entries" (aka Servlet) to menus (aka url patterns)
from a static file inside the war (web.xml). If it was not possible in 4
years to solve this problem in Eclipse, how will it be possible for Tomcat?

Damien

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[pkriens]

> To me, a webapp adds "entries" (aka Servlet) to menus (aka url patterns)
> from a static file inside the war (web.xml). If it was not possible in 4
> years to solve this problem in Eclipse, how will it be possible for Tomcat?
More and more code is supporting the dynamic life cycle model because it is
not that hard and the Eclipse people like it. However, you should realize that
there is a very diverse community out there. Only recently is OSGi on their
radar.

Providing a mechanism to get servlets registered from the web.xml are available.
The OPS 4J guys are providing tools and bundles: http://wiki.ops4j.org/confluence/display/ops4j/Pax+Web+Extender. It is actually not
that hard. However, if a web app creates resources, it also needs cleanup. Much of the
cleanup can be done on the server side because you can get an event when a bundle stops.
I.e. in the OSGi Http Service, it is required to automatically unregister a servlet when a
bundle is stopped. In some cases, the cleanup must be performed by the web app itself.
In those case, the web app must implement some code that is called when the bundle
is stopped.

This may all sound scary and complex, but it is surprisingly simple when you try it
out because it feels very natural in an OSGi environment. Programming models like Spring DM, iPOJO, DS, etc. can further help to support this model while not showing up in any Java code.

Kind regards,

   Peter Kriens
 

Damien Bonvillain wrote:

pkriens wrote:
>> Just my 2c: Eclipse, OSGified since 3.2, still doesn't fully support
>> updates without a JVM restart; and they have full control about the
>> whole aspects.
> Unfortunately there are tens of thousands of plugin writers out there that
> are not aware of the dynamics. Eclipse migrated to OSGi 4 years ago and did
> some incredible open heart surgery on their runtime engine to support OSGi.
> It was quite impressive how they replace their engine with an OSGi based
> version while existing plugins kept working. However, none of the existing
> code was ready for the consequences of dynamics. Plugins could add entries
> to menu's but this information came from a static file in the plugin, so
> there was no facility to remove this information.

To me, a webapp adds "entries" (aka Servlet) to menus (aka url patterns)
from a static file inside the war (web.xml). If it was not possible in 4
years to solve this problem in Eclipse, how will it be possible for Tomcat?

Damien

---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[Damien Bonvillain]

On Mon, Apr 28, 2008 at 12:52 PM, pkriens wrote:
> > To me, a webapp adds "entries" (aka Servlet) to menus (aka url patterns)
> > from a static file inside the war (web.xml). If it was not possible in 4
> > years to solve this problem in Eclipse, how will it be possible for
> > Tomcat?
> More and more code is supporting the dynamic life cycle model because it is
> not that hard and the Eclipse people like it. However, you should realize
> that there is a very diverse community out there. Only recently is OSGi on their
> radar.

I was talking about a very specific case; Eclipse 3.3, only with core
plugins, is still unable to be updated without a full restart. Or
maybe it's able to do it, but it does not know whether or not it's
safe to do it.

> Providing a mechanism to get servlets registered from the web.xml are
> available. The OPS 4J guys are providing tools and bundles:
> http://wiki.ops4j.org/confluence/display/ops4j/Pax+Web+Extender. It is
> actually not that hard.

So, it's easily doable, why didn't the Eclipse folks used that approach?
I see on the page you reference that JSPs must be registered manually,
and it works only with their specific HTTP Service Bundle:
"Then you have to let Pax Web know that you have jsps by calling the
registerJsps() method on WebContainer."
So basically, if you have a webapp with JSPs, with this "not hard too
write mechanism", you have too query a specific Bundle from the
webapp, and call a non-standard method. Am I reading that right?

> This may all sound scary and complex,

Please, I think the audience is pretty much "mature" developpers here :-)

> but it is surprisingly simple when you try it
> out because it feels very natural in an OSGi environment.

The goal is to make a .war to feel very comfortable in a Servlet
Container environment :-) If an action is required from the web
application builder, then bye bye RI status :-)
One thing I don't see in Pax Web Extender - War is the handling of
resource-env-ref; but I'm not sure it raises a real question though.

>  Programming models like Spring DM, iPOJO, DS, etc. can further help to support this model while
> not showing up in any Java code.

The good things when something shows up in Java code are that: it can
be easily be audited and it can be easily refactored. But maybe it's a
question of taste after all.

Cheers,
Damien

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...