Osgifing Tomcat

[pkriens]

>>  Please, I think the audience is pretty much "mature" developpers  
>> here :-)
I know, that is why it is scary and sounds complex because experienced  
developers
know much better what can go wrong. Novices have much fewer qualms. :-)

Anyway, the dynamism is not Eclipse's strongest feature, but they are  
getting better
because the underlying OSGi engine allows them to be dynamic. Update  
manager
now asks you if you want to restart, it used to just do it. And it  
often works. I think that it
just takes time for all the thousands of developers to get used to  
dynamics. There
might be similar issues with WARs but the key difference is, is that a  
WAR is
much more self contained than a plugin in Eclipse. Plugins willy nilly  
refer to each other.

I guess maybe it is time to gather some requirements for this effort?  
There are so
many ways to skin a cat, but you first need to know which cat. Because
I am not sure how big you want to make this effort? I do not know  
enough about
Tomcat to really make a lot of sense. Providing OSGi headers seems to  
fulfill
the immediate need of several groups. However, it would be really nice  
if you
could provide a service interface like an Http Service (Http Service  
itself is Servlet 2.1,
so you likely need something better). However, if you put the headers  
in place,
I am sure someone else will provide this functionality as another  
bundle.

I think the more ambitious case then is to leverage the OSGi class  
loading and
allow WARs to be installed dynamically. Several people (there is also an
RFC inside the OSGi about this, and we added several features to OSGi  
R4 for this
use case), are working on this. The devil is in the details, but I am  
pretty sure
you can do this with a bit of effort.

Of course then there is the possibility to turn Tomcat itself into  
separate bundles and
leverage OSGi internally. This is an interesting exercise but looking  
at the sentiments
in this list I think this is a far way off :-)

> So basically, if you have a webapp with JSPs, with this "not hard too
> write mechanism", you have too query a specific Bundle from the
> webapp, and call a non-standard method. Am I reading that right?

In OSGi, the bundles are first class citizens in the API. This allows an
extender bundle to read resources from the bundle when it gets started  
and
take appropriate actions. I.e. a WAR manager bundle can detect  
reliably the
installation of new bundles, read the appropriate XML files, and do the
registrations. So the "non standard" method would be internal Tomcat API
called by a Tomcat bundle. I think it is a very good requirement to  
keep the
WAR the WAR and not require any OSGi code in there. In the longer run
OSGi could update the rather stale Http Service specification to add  
this
kind of management API, allowing the WAR manager and the web server to  
be
independent bundles so that developers can mix and match.

I am not intricately involved with these efforts but I am more than
willing to organize some effort to get the different groups together  
that are
working on this?

Kind regards,

        Peter Kriens




On 28 apr 2008, at 23:23, Damien B wrote:

> On Mon, Apr 28, 2008 at 12:52 PM, pkriens wrote:
>>> To me, a webapp adds "entries" (aka Servlet) to menus (aka url  
>>> patterns)
>>> from a static file inside the war (web.xml). If it was not  
>>> possible in 4
>>> years to solve this problem in Eclipse, how will it be possible for
>>> Tomcat?
>> More and more code is supporting the dynamic life cycle model  
>> because it is
>> not that hard and the Eclipse people like it. However, you should  
>> realize
>> that there is a very diverse community out there. Only recently is  
>> OSGi on their
>> radar.
>
> I was talking about a very specific case; Eclipse 3.3, only with core
> plugins, is still unable to be updated without a full restart. Or
> maybe it's able to do it, but it does not know whether or not it's
> safe to do it.
>
>> Providing a mechanism to get servlets registered from the web.xml are
>> available. The OPS 4J guys are providing tools and bundles:
>> http://wiki.ops4j.org/confluence/display/ops4j/Pax+Web+Extender. It  
>> is
>> actually not that hard.
>
> So, it's easily doable, why didn't the Eclipse folks used that  
> approach?
> I see on the page you reference that JSPs must be registered manually,
> and it works only with their specific HTTP Service Bundle:
> "Then you have to let Pax Web know that you have jsps by calling the
> registerJsps() method on WebContainer."
> So basically, if you have a webapp with JSPs, with this "not hard too
> write mechanism", you have too query a specific Bundle from the
> webapp, and call a non-standard method. Am I reading that right?
>
>> This may all sound scary and complex,
>
> Please, I think the audience is pretty much "mature" developpers  
> here :-)
>
>> but it is surprisingly simple when you try it
>> out because it feels very natural in an OSGi environment.
>
> The goal is to make a .war to feel very comfortable in a Servlet
> Container environment :-) If an action is required from the web
> application builder, then bye bye RI status :-)
> One thing I don't see in Pax Web Extender - War is the handling of
> resource-env-ref; but I'm not sure it raises a real question though.
>
>> Programming models like Spring DM, iPOJO, DS, etc. can further help  
>> to support this model while
>> not showing up in any Java code.
>
> The good things when something shows up in Java code are that: it can
> be easily be audited and it can be easily refactored. But maybe it's a
> question of taste after all.
>
> Cheers,
> Damien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Costin Manolache-2]

On Mon, Apr 28, 2008 at 11:25 PM, Peter Kriens <peter.kriens@...>
wrote:
>  Tomcat to really make a lot of sense. Providing OSGi headers seems to
> fulfill
>  the immediate need of several groups. However, it would be really nice if
> you
>  could provide a service interface like an Http Service (Http Service
itself
> is Servlet 2.1,
>  so you likely need something better). However, if you put the headers in

Headers seems to be ok - I don't think anyone objected to this.

Regarding HttpService - I don't think it's a good idea for tomcat.
One of the major problems with OSGI ( and we need to make sure we don't fall
in this trap )
is the re-invention of common APIs - logging, servlet interfaces, etc. It
would be quite inappropriate
for tomcat to not use the standard deployment/configuration mechanism for
servlets.
So using or implementing any of the OSGI-re-defined service interfaces in
tomcat would be a hard sale IMO.


>  I think the more ambitious case then is to leverage the OSGi class
loading
> and
>  allow WARs to be installed dynamically. Several people (there is also an

There are 3 issues here:
- dynamic installation and uninstallation of WARs - that's already working
quite well, no need for OSGI.

- 'gapless' upgrade (old version running alongside with the new one)- OSGI
has no magic bullet for that, the problem
is more in the mapping layer.

- using OSGI class loading to allow more flexible interaction between WARs.
Unfortunately that is invalidated by
the servlet spec, which clearly defines this area.



>
>  Of course then there is the possibility to turn Tomcat itself into
separate
> bundles and
>  leverage OSGi internally. This is an interesting exercise but looking at
> the sentiments
>  in this list I think this is a far way off :-)

Quite the opposite, I think this would be the best use of OSGI - if we can
avoid the bad things in OSGI ( like the
service APIs and over-use of bundle activator and service registration ).

What tomcat needs the most IMO is a way to separate 'core' functionality
from 'modules' - like
authenticators, session management, connectors. Using OSGI bundles for
modules and integrating
with the lifecycle would be ok if it can be done non-intrusive:

- some OSGI-independent interface ( with a OSGI implementation and a
'direct' one ) to load the module.
I personally like JBoss style of modules as well - so that could be a 3rd
implementation ( maybe outside of apache ).

- some code loaded from server.xml that would start the OSGI container and
load the modules

- use OSGI/Jboss-style class loader to allow the modules and core to share
classes.

Most of this can be done in an 'optional' package ( i.e. in the trunk, but
not included in the 'official' base release ).
Few simple interfaces and impl changes may need to be added to the core -
but it can be very light.


>
>  In OSGi, the bundles are first class citizens in the API. This allows an
>  extender bundle to read resources from the bundle when it gets started
and
>  take appropriate actions. I.e. a WAR manager bundle can detect reliably
the
>  installation of new bundles, read the appropriate XML files, and do the
>  registrations. So the "non standard" method would be internal Tomcat API
>  called by a Tomcat bundle. I think it is a very good requirement to keep
> the
>  WAR the WAR and not require any OSGi code in there. In the longer run
>  OSGi could update the rather stale Http Service specification to add this
>  kind of management API, allowing the WAR manager and the web server to be
>  independent bundles so that developers can mix and match.

I'm pretty sure most people want to use OSGi for services, and Servlet API
for web development.
HttpService is a non-starter - it would be great if the next servlet API
defines interfaces
for dynamic (code-based) loading of servlets -  and we can provide some
tomcat-specific
interfaces ( like jetty does )  - but OSGi is not the right place to look
for servlet ( or logging ) APIs.

My main question regarding OSGI bundles and services:

Is there a way for a bundle to declare that it implements some services
using manifest or some config file, but without
 requiring it to be started to register services via BundleContext ? What I
would like is to start tomcat, use server.xml
to find what services are needed, and load the bundles providing the modules
( and only those ).


Costin

[hgomez]

Read on serverside that JSR-315 needs ideas for servlet 3.0 specs.

http://www.theserverside.com/news/thread.tss?thread_id=49212

May be a good opportunity to send various requests about dynamic
reload  purposes (with or without OSGI) to the JCR.

Regards

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Remy Maucherat]

On Tue, 2008-04-29 at 22:12 +0200, Henri Gomez wrote:
> Read on serverside that JSR-315 needs ideas for servlet 3.0 specs.

You should try to read the article I think :) This is about a specific
issue, where there's no actual disagreement (basically it is a publicity
stunt).

Rémy



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[hgomez]

>  You should try to read the article I think :) This is about a specific
>  issue, where there's no actual disagreement (basically it is a publicity
>  stunt).

I read it carefully Remy, don't worry.

There is open discussion on Servlet 3.0 and may be the opportunity to
discuss more than just discovering servlet from jars and using
annotations.

After all it's 3.0 rev and not 2.6, we could expect more revolution

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Costin Manolache-3]

Well, IMHO the servlet spec is going from bad to worse in terms of
complexity and feature bloat, so
careful what you wish :-)

My point was mostly that we don't have to implement OSGI HttpService, it may
be ok to use them for
modularization but for servlet-specific APIs we should stick with the JSRs.

I think extending a bit the JMX support ( i.e. exposing more setters and
methods ) would cover most
use cases and fit with the rest of tomcat quite well.

Costin

On Tue, Apr 29, 2008 at 1:12 PM, Henri Gomez <henri.gomez@...> wrote:

> Read on serverside that JSR-315 needs ideas for servlet 3.0 specs.
>
> http://www.theserverside.com/news/thread.tss?thread_id=49212
>
> May be a good opportunity to send various requests about dynamic
> reload  purposes (with or without OSGI) to the JCR.
>
> Regards
>

[pkriens]

> Regarding HttpService - I don't think it's a good idea for tomcat.
> One of the major problems with OSGI ( and we need to make sure we  
> don't fall
> in this trap ) is the re-invention of common APIs - logging, servlet  
> interfaces, etc.
As a bit of background. The logging and Http Service API are from  
1999. At that time
there was no dominant common logging API (neither in Java SE nor in  
open source),
and the Http Service API is 100% based on the, at that time, standard  
Servlet API (it
uses javax.servlet.http), it only provides an API to dynamically  
register and unregister
servlets, which is still lacking in the current Servlet API. On top of
that, all these service APIs are optional. If you look in more detail,  
than many services are
similar: providing a possibility to -dynamically- use Java APIs. I.e.  
XML parsers, IO Connectors
(J2ME), Servlets, URL stream and content handlers, Preferences, etc.  
We are not looking for
work and try to leverage the existing Java environment to the utmost  
extent.

This said, I agree that you want the core of a product like Tomcat to  
be as decoupled
as possible of any frameworks, including OSGi. Decoupling is the  
guiding principle behind
OSGi and the raison d'etre for most of its functionality. Not using a  
service is better than
coupling to it. But sometimes not using a service is more expensive  
than using it, that
decision is what design is all about.

> It would be quite inappropriate for tomcat to not use the standard  
> deployment/configuration mechanism for
> servlets. So using or implementing any of the OSGI-re-defined  
> service interfaces in
> tomcat would be a hard sale IMO.

Well, I do not see that this is an dichotomy. By nature of being the  
RI, you must follow the
JSR. However, it would not be hard to provide the Http Service as an  
additional component. If
Tomcat provides an API to dynamically register servlets, it would be  
trivial for someone
to provide an OSGi compatibility bundle, just like people are doing it  
today.
But it would be a nice service to get this from the horse's mouth. I  
am sure people are willing
to provide this code.

A bit more background. 4 years ago IBM started to like the OSGi  
modularity (class loaders on
steroids) and the silly, unnecessary life cycle and service layer.  
Actually, we had no visible
layers back then. At that time, there were raging debates about  
modularity with lots of
competition. This was actually the trigger for R4 to put the 4 layers  
into place: security, execution
environment, modularity, life cycle, service. Key idea was that people  
could pick the modularity
layer without dragging in the life cycle and service layer. I think  
the dislike for these
layers was based on lack of familiarity because what happened was that  
people started to
use the modularity, then found out how convenient the life cycle layer  
is in development. You
can keep your app server running while debugging and updating code.

However, getting hooked to the life cycle layer (if only for  
development) means things are
coming and going. Coupling through the module layer (i.e. factories,  
Class.forName, etc) means
you create import wires to other modules that can not be dynamically  
withdrawn because Java lacks an API for
this. In OSGi, it is not a big deal because you can still stop a  
bundle, update it, and refresh all the
import wires. This is feasible because bundles can be started and  
stopped and the OSGi framework
is intricately aware of these import wires.

In reality, this is a rather crude approach because in well designed  
systems the coupling between bundles
is minimal. At this point in time, services usually start to look more  
attractive because they provide
an API to allow dynamic updates without crudely stopping all bundles  
in the module dependency
graph (which in non-service based systems, and especially require-
bundle based systems tends
to become the whole system). And a service is just a POJO that is  
registered under one or more interfaces. By allowing
it to withdrawn at any moment in time, as well as registered by  
multiple independent parties, OSGi
provides a good abstraction of this dynamism. And there is no Java  
counterpart for this.

With JSR 291 we had the same situation. There was a requirement to get  
rid of the service layer
because it was deemed unnecessary. We puzzled with the interfaces in  
OSGi so that we
could be backward compatible and still have this separation. We got  
some solution but it
felt awkward. Worse, there are a number of optional APIs in the OSGi  
framework to manage
the modularity layer (PackageAdmin), the security (PermissionAdmin and  
ConditionalPermissionAdmin),
startlevels, and URL stream and content handling. By removing the  
service layer we had to
find an alternative way to provide these APIs to the bundles.  
Factories? Dependency Injection?
Class.forName? They all felt far inferior to the service model, all  
lacking the dynamics. We then
looked at the API, only 4 specific classes and 3 methods on the core  
framework interface class
BundleContext. The implementations were around 30-40k so in the end we  
decided to keep the
service layer around in JSR 291 because there was no decent  
alternative, the functionality
was needed, and 50k seems a low price to pay.

Last illustration and then I shut up. 2 years ago we were approached  
by Spring. Customers had told them that OSGi was
cool. Spring is in the business of providing convenience APIs to  
complex core APIs and they approached
this job in that vein. OSGi, just one of the zillion environments that  
had to be supported
by the Spring suite. However, something very interesting happened over  
the next few months. I think
they fell in love and the service layer was a major part of their  
infatuation. They
realized very quickly how they could leverage the services as beans in  
their model and the
advantages of dynamism without rebooting.

> Is there a way for a bundle to declare that it implements some  
> services
> using manifest or some config file, but without requiring it to be  
> started to register
> services via BundleContext ? What I would like is to start tomcat,  
> use server.xml
> to find what services are needed, and load the bundles providing the  
> modules
> ( and only those ).
This is the use case of Declarative Services. The Service Component  
Runtime (SCR)
will inspect the bundle when it gets installed and register any  
services that
are declared and which have their dependencies on other services  
satisfied.

The bundles themselves are not having a classloader, nor started. Only  
when the
service is really used by another bundle will the bundle be activated  
by the
SCR. If the dependencies go away, the component is deactivated. This  
is of course
all runtime based.

There is also OBR (Apache Felix's Richard Hall is the key author of  
this) which is the
OSGi Bundle Repository. This is a model where you can install bundle  
X, and then
OBR can install any dependent bundles. The dependency model is very  
flexible (an extension
to JSR 124) which also supports dependencies on services. However,  
realize that the dynamic
nature means that there never is a guarantee a service is registered  
by a bundle, even
if it is started because some dependency can be missing, or a bundle  
can be updated, etc.

The OSGi manifest does have a header Export-Service and Import-Service  
defined but because
you can't give an guarantees about them without running the bundle, we  
decided to deprecate them.

However, how much do you want to be in the business of deployment?  
Tomcat is very often used
as an application server but the web site tells me your core is a  
Servlet and
JSP implementation. If you are going for strict modularity, wouldn't  
it make sense to distinguish
between these core functions; Aren't the application management  
aspects secondary? I.e.
allow your core JSP and Servlet engines to be used by others that  
provide the deployment
aspects and allow your deployment code to be able to use other web  
service providers? How
can you leverage existing deployment servers and management agents?  
For me the key thing about
modularity is to do as little as possible and focus on core business.

Sorry for the rather verbose mail, I did not have enough time to write  
a short one. :-(

Kind regards,

        Peter Kriens







On 29 apr 2008, at 19:48, Costin Manolache wrote:

> On Mon, Apr 28, 2008 at 11:25 PM, Peter Kriens  
> <peter.kriens@...>
> wrote:
>> Tomcat to really make a lot of sense. Providing OSGi headers seems to
>> fulfill
>> the immediate need of several groups. However, it would be really  
>> nice if
>> you
>> could provide a service interface like an Http Service (Http Service
> itself
>> is Servlet 2.1,
>> so you likely need something better). However, if you put the  
>> headers in
>
> Headers seems to be ok - I don't think anyone objected to this.
>
> Regarding HttpService - I don't think it's a good idea for tomcat.
> One of the major problems with OSGI ( and we need to make sure we  
> don't fall
> in this trap )
> is the re-invention of common APIs - logging, servlet interfaces,  
> etc. It
> would be quite inappropriate
> for tomcat to not use the standard deployment/configuration  
> mechanism for
> servlets.
> So using or implementing any of the OSGI-re-defined service  
> interfaces in
> tomcat would be a hard sale IMO.
>
>
>> I think the more ambitious case then is to leverage the OSGi class
> loading
>> and
>> allow WARs to be installed dynamically. Several people (there is  
>> also an
>
> There are 3 issues here:
> - dynamic installation and uninstallation of WARs - that's already  
> working
> quite well, no need for OSGI.
>
> - 'gapless' upgrade (old version running alongside with the new  
> one)- OSGI
> has no magic bullet for that, the problem
> is more in the mapping layer.
>
> - using OSGI class loading to allow more flexible interaction  
> between WARs.
> Unfortunately that is invalidated by
> the servlet spec, which clearly defines this area.
>
>
>
>>
>> Of course then there is the possibility to turn Tomcat itself into
> separate
>> bundles and
>> leverage OSGi internally. This is an interesting exercise but  
>> looking at
>> the sentiments
>> in this list I think this is a far way off :-)
>
> Quite the opposite, I think this would be the best use of OSGI - if  
> we can
> avoid the bad things in OSGI ( like the
> service APIs and over-use of bundle activator and service  
> registration ).
>
> What tomcat needs the most IMO is a way to separate 'core'  
> functionality
> from 'modules' - like
> authenticators, session management, connectors. Using OSGI bundles for
> modules and integrating
> with the lifecycle would be ok if it can be done non-intrusive:
>
> - some OSGI-independent interface ( with a OSGI implementation and a
> 'direct' one ) to load the module.
> I personally like JBoss style of modules as well - so that could be  
> a 3rd
> implementation ( maybe outside of apache ).
>
> - some code loaded from server.xml that would start the OSGI  
> container and
> load the modules
>
> - use OSGI/Jboss-style class loader to allow the modules and core to  
> share
> classes.
>
> Most of this can be done in an 'optional' package ( i.e. in the  
> trunk, but
> not included in the 'official' base release ).
> Few simple interfaces and impl changes may need to be added to the  
> core -
> but it can be very light.
>
>
>>
>> In OSGi, the bundles are first class citizens in the API. This  
>> allows an
>> extender bundle to read resources from the bundle when it gets  
>> started
> and
>> take appropriate actions. I.e. a WAR manager bundle can detect  
>> reliably
> the
>> installation of new bundles, read the appropriate XML files, and do  
>> the
>> registrations. So the "non standard" method would be internal  
>> Tomcat API
>> called by a Tomcat bundle. I think it is a very good requirement to  
>> keep
>> the
>> WAR the WAR and not require any OSGi code in there. In the longer run
>> OSGi could update the rather stale Http Service specification to  
>> add this
>> kind of management API, allowing the WAR manager and the web server  
>> to be
>> independent bundles so that developers can mix and match.
>
> I'm pretty sure most people want to use OSGi for services, and  
> Servlet API
> for web development. HttpService is a non-starter - it would be  
> great if the next servlet API
> defines interfaces for dynamic (code-based) loading of servlets -  
> and we can provide some
> tomcat-specific interfaces ( like jetty does )  - but OSGi is not  
> the right place to look
> for servlet ( or logging ) APIs.
>
> My main question regarding OSGI bundles and services:
>
> Is there a way for a bundle to declare that it implements some  
> services
> using manifest or some config file, but without
> requiring it to be started to register services via BundleContext ?  
> What I
> would like is to start tomcat, use server.xml
> to find what services are needed, and load the bundles providing the  
> modules
> ( and only those ).
>
>
> Costin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Niall Pemberton-2]

On Tue, Apr 22, 2008 at 11:45 AM, Henri Gomez <henri.gomez@...> wrote:
> Hi to all,
>
>  Did there is plans, ideas or interest around about OSGI-fing Tomcat ?

Quotes from http://www.infoq.com/news/2008/04/springsource-app-platform

"...the SpringSource Application Platform, an application server built
on Spring, OSGi, and Apache Tomcat"

"Tomcat is included as an OSGi bundle to support web functionality."

"SpringSource will also be submitting patches back to projects such as
Tomcat for any changes they have made to enable a library to be OSGi
bundle compatible."

Niall

>  Regards

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Costin Manolache-2]

On Wed, Apr 30, 2008 at 1:00 AM, Peter Kriens <peter.kriens@...>
wrote:

> Regarding HttpService - I don't think it's a good idea for tomcat.
> > One of the major problems with OSGI ( and we need to make sure we don't
> > fall
> > in this trap ) is the re-invention of common APIs - logging, servlet
> > interfaces, etc.
> >
> As a bit of background. The logging and Http Service API are from 1999. At
> that time
> there was no dominant common logging API (neither in Java SE nor in open
> source),
> and the Http Service API is 100% based on the, at that time, standard
> Servlet API (it
> uses javax.servlet.http), it only provides an API to dynamically register
> and unregister
> servlets, which is still lacking in the current Servlet API.

Regarding 'dynamic register/unregister' - the servlet API defines one way to
do this, i.e. war and the
deployment. There is no standard API to install/uninstall/start/stop a .war
- but HttpService
is not that either. Runtime config changes ( adding/removing servlets
without web.xml changes
and re-deployment ) is not specified, but it's a whole different discussion
for the JSR people
 to solve, I'm pretty sure it'll be different from HttpService.



 It would be quite inappropriate for tomcat to not use the standard

> > deployment/configuration mechanism for
> > servlets. So using or implementing any of the OSGI-re-defined service
> > interfaces in
> > tomcat would be a hard sale IMO.
> >
>
> Well, I do not see that this is an dichotomy. By nature of being the RI,
> you must follow the
> JSR. However, it would not be hard to provide the Http Service as an
> additional component. If
> Tomcat provides an API to dynamically register servlets, it would be
> trivial for someone
> to provide an OSGi compatibility bundle, just like people are doing it
> today.
> But it would be a nice service to get this from the horse's mouth. I am
> sure people are willing
> to provide this code.
>

A lot of people would like for tomcat to provide more jetty-like APIs for
programmatic
configuration ( and in a way it is already possible - just not easy ).
As an API, HttpService is way off - in '99 and servlet 2.1 it may have been
valuable.

Let's keep HttpService for a different discussion :-)

> In reality, this is a rather crude approach because in well designed
> systems the coupling between bundles
> is minimal. At this point in time, services usually start to look more
> attractive because they provide
> an API to allow dynamic updates without crudely stopping all bundles in
> the module dependency
> graph (which in non-service based systems, and especially require-bundle
> based systems tends
> to become the whole system). And a service is just a POJO that is
> registered under one or more interfaces. By allowing
> it to withdrawn at any moment in time, as well as registered by multiple
> independent parties, OSGi
> provides a good abstraction of this dynamism. And there is no Java
> counterpart for this.
>

Actually - JMX provides a lot of this dynamism.  Tomcat is using a lot of
JMX ( and hopefully will use more ),
and provide very similar model with OSGI services.





> they fell in love and the service layer was a major part of their
> infatuation. They
> realized very quickly how they could leverage the services as beans in
> their model and the
> advantages of dynamism without rebooting.


To clarify: updating webapps in tomcat without rebooting has been around for
many years :-).
Webapps are quite similar with the bundles - it would be interesting to see
if we could use
OSGI classloader instead of ours, but I don't think that's a real problem.
People are looking for 'gapless' restart, i.e. users who have an active
session continuing to
use the old version ( or some magic way to migrate the session - but that's
likely impossible
in most real cases, i.e. if code changes happen ). OSGi alone can't solve
this.

A whole different class of users would like to see _less_ dynamism - i.e.
embedding tomcat
as a jar and using simple APIs and no class loaders or config files. In both
high-end servers
and low-end embedding this is a very important use case.


The problem I would like to see solved in tomcat is breaking it up in
modules - i.e. separating
all the optional parts and having a way to ship a really minimal core and a
good way to release and
deploy add-on bundles. OSGi may be a good way to do this.

I think the requirements are:
- provide a way for different tomcat components and core to be packaged as
OSGi bundles, using manifests
and maybe optional activators if it can't be avoided ( as long as tomcat
doesn't depend on it ). I think this
is an easy sell, I can't think of any good technical reasons not to do it.

- break tomcat release ( for 7.0 or 6.5 or whatever trunk will be released
as ) into just core ( minimal servlet
impl ) and bundles

- find a easy way for people to download/install all modules they want.

- integrate this with the JMX layer and the manager servlet

- all this without a hard dependency on OSGI - for people who want 'one big
jar' for embedding
( we don't want to force them to use osgi - or be dynamic - if they don't
have a need for this)

I think 'more dynamism' is a good thing - but so is 'less dynamism',  both
are valid  goals and we  can and
should support both. ( right now we're somewhere in the middle -  don't
really support either one well enough )


 Is there a way for a bundle to declare that it implements some services
> using manifest or some config file, but without requiring it to be started
> to register
> services via BundleContext ? What I would like is to start tomcat, use
> server.xml
> to find what services are needed, and load the bundles providing the
> modules
> ( and only those ).
>
This is the use case of Declarative Services. The Service Component Runtime
> (SCR)
> will inspect the bundle when it gets installed and register any services
> that
> are declared and which have their dependencies on other services
> satisfied.


Any pointer to the docs ?

If it only involves declarative stuff ( xml, manifest ) and is done at
install time - it may work
for us.

Having a bundle start just to declare they provide a service (that won't be
used) is a
non-starter.

The current model in tomcat is to use Class.forName to load components using
server.xml -
only the code that is used is started. I don't think it'll be acceptable to
replace
server.xml or use OSGi for configuration ( in particular when JMX is a
better solution for
many config problems ).



> The OSGi manifest does have a header Export-Service and Import-Service
> defined but because
> you can't give an guarantees about them without running the bundle, we
> decided to deprecate them.


Too bad :-)


>
> However, how much do you want to be in the business of deployment? Tomcat
> is very often used
> as an application server but the web site tells me your core is a Servlet
> and
> JSP implementation.


Tomcat is in the business of deployment and configuration of WAR files -
that's a core part of the spec.

> If you are going for strict modularity, wouldn't it make sense to
> distinguish
> between these core functions; Aren't the application management aspects
> secondary? I.e.
> allow your core JSP and Servlet engines to be used by others that provide
> the deployment
> aspects and allow your deployment code to be able to use other web service
> providers? How
> can you leverage existing deployment servers and management agents? For me
> the key thing about
> modularity is to do as little as possible and focus on core business.

Sure - the core business of tomcat is to deploy/configure/run WAR files
conforming to the spec.


The biggest problems ( IMO ) facing tomcat are:
- it needs more modularity in terms of its own extensions ( connectors,
auth, session managers, etc ).
- it needs to be easier to embed - both dynamically and as 'one big jar' (
or jetty-style ).
For both OSGi could help.

( well, we also have some social problems from time to time :-)

Costin

[djencks]

On Apr 30, 2008, at 10:28 AM, Costin Manolache wrote:

> On Wed, Apr 30, 2008 at 1:00 AM, Peter Kriens <peter.kriens@...>
> wrote:
>
>> Regarding HttpService - I don't think it's a good idea for tomcat.
>>> One of the major problems with OSGI ( and we need to make sure we  
>>> don't
>>> fall
>>> in this trap ) is the re-invention of common APIs - logging, servlet
>>> interfaces, etc.
>>>
>> As a bit of background. The logging and Http Service API are from  
>> 1999. At
>> that time
>> there was no dominant common logging API (neither in Java SE nor in  
>> open
>> source),
>> and the Http Service API is 100% based on the, at that time, standard
>> Servlet API (it
>> uses javax.servlet.http), it only provides an API to dynamically  
>> register
>> and unregister
>> servlets, which is still lacking in the current Servlet API.
>
>
>
> Regarding 'dynamic register/unregister' - the servlet API defines  
> one way to
> do this, i.e. war and the
> deployment. There is no standard API to install/uninstall/start/stop  
> a .war

umm, jsr-88??

david jencks

>
> - but HttpService
> is not that either. Runtime config changes ( adding/removing servlets
> without web.xml changes
> and re-deployment ) is not specified, but it's a whole different  
> discussion
> for the JSR people
> to solve, I'm pretty sure it'll be different from HttpService.
>
>
>
> It would be quite inappropriate for tomcat to not use the standard
>>> deployment/configuration mechanism for
>>> servlets. So using or implementing any of the OSGI-re-defined  
>>> service
>>> interfaces in
>>> tomcat would be a hard sale IMO.
>>>
>>
>> Well, I do not see that this is an dichotomy. By nature of being  
>> the RI,
>> you must follow the
>> JSR. However, it would not be hard to provide the Http Service as an
>> additional component. If
>> Tomcat provides an API to dynamically register servlets, it would be
>> trivial for someone
>> to provide an OSGi compatibility bundle, just like people are doing  
>> it
>> today.
>> But it would be a nice service to get this from the horse's mouth.  
>> I am
>> sure people are willing
>> to provide this code.
>>
>
> A lot of people would like for tomcat to provide more jetty-like  
> APIs for
> programmatic
> configuration ( and in a way it is already possible - just not easy ).
> As an API, HttpService is way off - in '99 and servlet 2.1 it may  
> have been
> valuable.
>
> Let's keep HttpService for a different discussion :-)
>
>
>
>
>
>> In reality, this is a rather crude approach because in well designed
>> systems the coupling between bundles
>> is minimal. At this point in time, services usually start to look  
>> more
>> attractive because they provide
>> an API to allow dynamic updates without crudely stopping all  
>> bundles in
>> the module dependency
>> graph (which in non-service based systems, and especially require-
>> bundle
>> based systems tends
>> to become the whole system). And a service is just a POJO that is
>> registered under one or more interfaces. By allowing
>> it to withdrawn at any moment in time, as well as registered by  
>> multiple
>> independent parties, OSGi
>> provides a good abstraction of this dynamism. And there is no Java
>> counterpart for this.
>>
>
> Actually - JMX provides a lot of this dynamism.  Tomcat is using a  
> lot of
> JMX ( and hopefully will use more ),
> and provide very similar model with OSGI services.
>
>
>
>
>
>> they fell in love and the service layer was a major part of their
>> infatuation. They
>> realized very quickly how they could leverage the services as beans  
>> in
>> their model and the
>> advantages of dynamism without rebooting.
>
>
> To clarify: updating webapps in tomcat without rebooting has been  
> around for
> many years :-).
> Webapps are quite similar with the bundles - it would be interesting  
> to see
> if we could use
> OSGI classloader instead of ours, but I don't think that's a real  
> problem.
> People are looking for 'gapless' restart, i.e. users who have an  
> active
> session continuing to
> use the old version ( or some magic way to migrate the session - but  
> that's
> likely impossible
> in most real cases, i.e. if code changes happen ). OSGi alone can't  
> solve
> this.
>
> A whole different class of users would like to see _less_ dynamism -  
> i.e.
> embedding tomcat
> as a jar and using simple APIs and no class loaders or config files.  
> In both
> high-end servers
> and low-end embedding this is a very important use case.
>
>
> The problem I would like to see solved in tomcat is breaking it up in
> modules - i.e. separating
> all the optional parts and having a way to ship a really minimal  
> core and a
> good way to release and
> deploy add-on bundles. OSGi may be a good way to do this.
>
> I think the requirements are:
> - provide a way for different tomcat components and core to be  
> packaged as
> OSGi bundles, using manifests
> and maybe optional activators if it can't be avoided ( as long as  
> tomcat
> doesn't depend on it ). I think this
> is an easy sell, I can't think of any good technical reasons not to  
> do it.
>
> - break tomcat release ( for 7.0 or 6.5 or whatever trunk will be  
> released
> as ) into just core ( minimal servlet
> impl ) and bundles
>
> - find a easy way for people to download/install all modules they  
> want.
>
> - integrate this with the JMX layer and the manager servlet
>
> - all this without a hard dependency on OSGI - for people who want  
> 'one big
> jar' for embedding
> ( we don't want to force them to use osgi - or be dynamic - if they  
> don't
> have a need for this)
>
> I think 'more dynamism' is a good thing - but so is 'less  
> dynamism',  both
> are valid  goals and we  can and
> should support both. ( right now we're somewhere in the middle -  
> don't
> really support either one well enough )
>
>
> Is there a way for a bundle to declare that it implements some  
> services
>> using manifest or some config file, but without requiring it to be  
>> started
>> to register
>> services via BundleContext ? What I would like is to start tomcat,  
>> use
>> server.xml
>> to find what services are needed, and load the bundles providing the
>> modules
>> ( and only those ).
>>
> This is the use case of Declarative Services. The Service Component  
> Runtime
>> (SCR)
>> will inspect the bundle when it gets installed and register any  
>> services
>> that
>> are declared and which have their dependencies on other services
>> satisfied.
>
>
> Any pointer to the docs ?
>
> If it only involves declarative stuff ( xml, manifest ) and is done at
> install time - it may work
> for us.
>
> Having a bundle start just to declare they provide a service (that  
> won't be
> used) is a
> non-starter.
>
> The current model in tomcat is to use Class.forName to load  
> components using
> server.xml -
> only the code that is used is started. I don't think it'll be  
> acceptable to
> replace
> server.xml or use OSGi for configuration ( in particular when JMX is a
> better solution for
> many config problems ).
>
>
>
>> The OSGi manifest does have a header Export-Service and Import-
>> Service
>> defined but because
>> you can't give an guarantees about them without running the bundle,  
>> we
>> decided to deprecate them.
>
>
> Too bad :-)
>
>
>>
>> However, how much do you want to be in the business of deployment?  
>> Tomcat
>> is very often used
>> as an application server but the web site tells me your core is a  
>> Servlet
>> and
>> JSP implementation.
>
>
> Tomcat is in the business of deployment and configuration of WAR  
> files -
> that's a core part of the spec.
>
>
>
>> If you are going for strict modularity, wouldn't it make sense to
>> distinguish
>> between these core functions; Aren't the application management  
>> aspects
>> secondary? I.e.
>> allow your core JSP and Servlet engines to be used by others that  
>> provide
>> the deployment
>> aspects and allow your deployment code to be able to use other web  
>> service
>> providers? How
>> can you leverage existing deployment servers and management agents?  
>> For me
>> the key thing about
>> modularity is to do as little as possible and focus on core business.
>
>
> Sure - the core business of tomcat is to deploy/configure/run WAR  
> files
> conforming to the spec.
>
>
> The biggest problems ( IMO ) facing tomcat are:
> - it needs more modularity in terms of its own extensions  
> ( connectors,
> auth, session managers, etc ).
> - it needs to be easier to embed - both dynamically and as 'one big  
> jar' (
> or jetty-style ).
> For both OSGi could help.
>
> ( well, we also have some social problems from time to time :-)
>
> Costin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[pkriens]

> Regarding 'dynamic register/unregister' - the servlet API defines  
> one way to
> do this, i.e. war and the deployment. There is no standard API to  
> install/uninstall/start/stop a .war
> - but HttpService is not that either. Runtime config changes  
> ( adding/removing servlets
> without web.xml changes and re-deployment ) is not specified, but  
> it's a whole different discussion
> for the JSR people to solve, I'm pretty sure it'll be different from  
> HttpService.

Ok, but who is re-inventing here? :-)

> Actually - JMX provides a lot of this dynamism.  Tomcat is using a  
> lot of
> JMX ( and hopefully will use more ), and provide very similar model  
> with OSGI services.
I think you underestimate what OSGi does and how closely the module  
layer, lifecycle layer,
and service layer interact to make things work for the programmer.  
OSGi is really more than
classloaders on steroids.

However, referring to an earlier  discussion, I think the trick is not  
to become dependent on
any one technology. I.e. the core Servlet and JSP engines should be  
POJO, and receive their
connections from the outside. Be that JMX, OSGi, or whatever. This  
would make it trivial
to provide the core functionality as a bean as well as an OSGi  
service. As long as the engines
can handle the dynamism.

The OSGi framework is about one idea: decoupling. All OSGi services  
are optional and none of the
services is based on other services. The majority of bundles does not  
have to touch OSGi API. The
recommendation has always been to configure your bundle in the bundle  
activator, but do not perform
any user functionality there so you can reuse your implementations in  
other contexts.

The OSGi functionality is limited to the management of the environment  
and how these decoupled units
can leverage each other. Interestingly, the majority of open source  
projects have (re)invented this
wheel over and over again, invariably not taking into account the  
important issues like versioning
and strict modularity.

> Webapps are quite similar with the bundles - it would be interesting  
> to see
> if we could use OSGI classloader instead of ours, but I don't think  
> that's a real problem.
Well, you seem to have the desired functionality? Why switch? I would  
never update a project
to use a new technology until it really is a significant improvement.  
I do not think the value
proposition of the OSGi class loaders warrants the switch on its own?

> People are looking for 'gapless' restart, i.e. users who have an  
> active
> session continuing to use the old version ( or some magic way to  
> migrate the session - but that's
> likely impossible in most real cases, i.e. if code changes happen ).  
> OSGi alone can't solve
> this.
Not on its own. However, the bundle model allows you to do a lot of  
interesting things here. In the
Http Service model you know exactly the bundle that registered the  
servlet (through the use of an
OSGi Service Factory). If the servlet is unregistered, I assume you  
can keep the session object around.
the problem is, I am out of my league here, is that the session object  
contains classes
to the old bundle. The trick is to convert these to the classes in the  
updated bundle. I guess you
could use serialization (you know exactly the bundle where the new  
classes should come from) or a fancy
reflection based mechanism, or use a mechanism that includes the bundle.
Then again, this is not my area of expertise. However, I do believe that
the bundle model and service model with their notification model do  
help allow you to do this
without ending up in class cast exceptions and version mismatches.

Of course there are nasty issues when classes in a session are reused  
by independent bundles. We have
a similar (I think) problem with security. One bundle can use a  
different version of a permission class
than another. We therefore convert the permission info to the correct  
version of the permission
class when doing the check because we then know exactly what version  
that caller is using. You could
do something similar with the session object. If an object is gotten  
from it, you can check if the servlet's
bundle is compatible with the class in the session and automatically  
convert it to the appropriate class.

Then again, I am not an expert here so this might not be the problem  
at all. I only know that the devil
is in the details. :-(

> A whole different class of users would like to see _less_ dynamism -  
> i.e.
> embedding tomcat as a jar and using simple APIs and no class loaders  
> or config files. In both
> high-end servers and low-end embedding this is a very important use  
> case.
I agree, this is aligned with the POJO ideas. Separate the  
functionality in a set of completely disjoint
functionalities. Then allow the customer to wire them into an OSGi  
based system, JMX based system,
standalone server, etc. You can then always provide one or more of  
those instances. The trick is
decoupling functional units from each other, with requires that things  
that are put together are
highly cohesive. Anyway, no dynamism is the simple instance of  
dynamism ...

I think the remainder of your mail is about this decomposition into  
functional units. I wholeheartedly
agree that you want to minimize dependencies on OSGi API. In this  
phase I would need a whiteboard
with people that have expertise in the different units to make the  
best decomposition. I will be at JavaOne next
week so if people want to discuss the use of OSGi, feel free to  
contact me there.

> ( well, we also have some social problems from time to time :-)
I must admit I feel I am walking on eggs ... and I am a bit surprised  
how few others tune in. If I am
trespassing here, please let me know kindly :-)

Kind regards,

        Peter Kriens







On 30 apr 2008, at 19:28, Costin Manolache wrote:

> On Wed, Apr 30, 2008 at 1:00 AM, Peter Kriens <peter.kriens@...>
> wrote:
>
>> Regarding HttpService - I don't think it's a good idea for tomcat.
>>> One of the major problems with OSGI ( and we need to make sure we  
>>> don't
>>> fall
>>> in this trap ) is the re-invention of common APIs - logging, servlet
>>> interfaces, etc.
>>>
>> As a bit of background. The logging and Http Service API are from  
>> 1999. At
>> that time
>> there was no dominant common logging API (neither in Java SE nor in  
>> open
>> source),
>> and the Http Service API is 100% based on the, at that time, standard
>> Servlet API (it
>> uses javax.servlet.http), it only provides an API to dynamically  
>> register
>> and unregister
>> servlets, which is still lacking in the current Servlet API.
>
>
>
> Regarding 'dynamic register/unregister' - the servlet API defines  
> one way to
> do this, i.e. war and the
> deployment. There is no standard API to install/uninstall/start/stop  
> a .war
> - but HttpService
> is not that either. Runtime config changes ( adding/removing servlets
> without web.xml changes
> and re-deployment ) is not specified, but it's a whole different  
> discussion
> for the JSR people
> to solve, I'm pretty sure it'll be different from HttpService.
>
>
>
> It would be quite inappropriate for tomcat to not use the standard
>>> deployment/configuration mechanism for
>>> servlets. So using or implementing any of the OSGI-re-defined  
>>> service
>>> interfaces in
>>> tomcat would be a hard sale IMO.
>>>
>>
>> Well, I do not see that this is an dichotomy. By nature of being  
>> the RI,
>> you must follow the
>> JSR. However, it would not be hard to provide the Http Service as an
>> additional component. If
>> Tomcat provides an API to dynamically register servlets, it would be
>> trivial for someone
>> to provide an OSGi compatibility bundle, just like people are doing  
>> it
>> today.
>> But it would be a nice service to get this from the horse's mouth.  
>> I am
>> sure people are willing
>> to provide this code.
>>
>
> A lot of people would like for tomcat to provide more jetty-like  
> APIs for
> programmatic
> configuration ( and in a way it is already possible - just not easy ).
> As an API, HttpService is way off - in '99 and servlet 2.1 it may  
> have been
> valuable.
>
> Let's keep HttpService for a different discussion :-)
>
>
>
>
>
>> In reality, this is a rather crude approach because in well designed
>> systems the coupling between bundles
>> is minimal. At this point in time, services usually start to look  
>> more
>> attractive because they provide
>> an API to allow dynamic updates without crudely stopping all  
>> bundles in
>> the module dependency
>> graph (which in non-service based systems, and especially require-
>> bundle
>> based systems tends
>> to become the whole system). And a service is just a POJO that is
>> registered under one or more interfaces. By allowing
>> it to withdrawn at any moment in time, as well as registered by  
>> multiple
>> independent parties, OSGi
>> provides a good abstraction of this dynamism. And there is no Java
>> counterpart for this.
>>
>
> Actually - JMX provides a lot of this dynamism.  Tomcat is using a  
> lot of
> JMX ( and hopefully will use more ),
> and provide very similar model with OSGI services.
>
>
>
>
>
>> they fell in love and the service layer was a major part of their
>> infatuation. They
>> realized very quickly how they could leverage the services as beans  
>> in
>> their model and the
>> advantages of dynamism without rebooting.
>
>
> To clarify: updating webapps in tomcat without rebooting has been  
> around for
> many years :-).
> Webapps are quite similar with the bundles - it would be interesting  
> to see
> if we could use
> OSGI classloader instead of ours, but I don't think that's a real  
> problem.
> People are looking for 'gapless' restart, i.e. users who have an  
> active
> session continuing to
> use the old version ( or some magic way to migrate the session - but  
> that's
> likely impossible
> in most real cases, i.e. if code changes happen ). OSGi alone can't  
> solve
> this.
>
> A whole different class of users would like to see _less_ dynamism -  
> i.e.
> embedding tomcat
> as a jar and using simple APIs and no class loaders or config files.  
> In both
> high-end servers
> and low-end embedding this is a very important use case.
>
>
> The problem I would like to see solved in tomcat is breaking it up in
> modules - i.e. separating
> all the optional parts and having a way to ship a really minimal  
> core and a
> good way to release and
> deploy add-on bundles. OSGi may be a good way to do this.
>
> I think the requirements are:
> - provide a way for different tomcat components and core to be  
> packaged as
> OSGi bundles, using manifests
> and maybe optional activators if it can't be avoided ( as long as  
> tomcat
> doesn't depend on it ). I think this
> is an easy sell, I can't think of any good technical reasons not to  
> do it.
>
> - break tomcat release ( for 7.0 or 6.5 or whatever trunk will be  
> released
> as ) into just core ( minimal servlet
> impl ) and bundles
>
> - find a easy way for people to download/install all modules they  
> want.
>
> - integrate this with the JMX layer and the manager servlet
>
> - all this without a hard dependency on OSGI - for people who want  
> 'one big
> jar' for embedding
> ( we don't want to force them to use osgi - or be dynamic - if they  
> don't
> have a need for this)
>
> I think 'more dynamism' is a good thing - but so is 'less  
> dynamism',  both
> are valid  goals and we  can and
> should support both. ( right now we're somewhere in the middle -  
> don't
> really support either one well enough )
>
>
> Is there a way for a bundle to declare that it implements some  
> services
>> using manifest or some config file, but without requiring it to be  
>> started
>> to register
>> services via BundleContext ? What I would like is to start tomcat,  
>> use
>> server.xml
>> to find what services are needed, and load the bundles providing the
>> modules
>> ( and only those ).
>>
> This is the use case of Declarative Services. The Service Component  
> Runtime
>> (SCR)
>> will inspect the bundle when it gets installed and register any  
>> services
>> that
>> are declared and which have their dependencies on other services
>> satisfied.
>
>
> Any pointer to the docs ?
>
> If it only involves declarative stuff ( xml, manifest ) and is done at
> install time - it may work
> for us.
>
> Having a bundle start just to declare they provide a service (that  
> won't be
> used) is a
> non-starter.
>
> The current model in tomcat is to use Class.forName to load  
> components using
> server.xml -
> only the code that is used is started. I don't think it'll be  
> acceptable to
> replace
> server.xml or use OSGi for configuration ( in particular when JMX is a
> better solution for
> many config problems ).
>
>
>
>> The OSGi manifest does have a header Export-Service and Import-
>> Service
>> defined but because
>> you can't give an guarantees about them without running the bundle,  
>> we
>> decided to deprecate them.
>
>
> Too bad :-)
>
>
>>
>> However, how much do you want to be in the business of deployment?  
>> Tomcat
>> is very often used
>> as an application server but the web site tells me your core is a  
>> Servlet
>> and
>> JSP implementation.
>
>
> Tomcat is in the business of deployment and configuration of WAR  
> files -
> that's a core part of the spec.
>
>
>
>> If you are going for strict modularity, wouldn't it make sense to
>> distinguish
>> between these core functions; Aren't the application management  
>> aspects
>> secondary? I.e.
>> allow your core JSP and Servlet engines to be used by others that  
>> provide
>> the deployment
>> aspects and allow your deployment code to be able to use other web  
>> service
>> providers? How
>> can you leverage existing deployment servers and management agents?  
>> For me
>> the key thing about
>> modularity is to do as little as possible and focus on core business.
>
>
> Sure - the core business of tomcat is to deploy/configure/run WAR  
> files
> conforming to the spec.
>
>
> The biggest problems ( IMO ) facing tomcat are:
> - it needs more modularity in terms of its own extensions  
> ( connectors,
> auth, session managers, etc ).
> - it needs to be easier to embed - both dynamically and as 'one big  
> jar' (
> or jetty-style ).
> For both OSGi could help.
>
> ( well, we also have some social problems from time to time :-)
>
> Costin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Filip Hanik - Dev Lists]

Peter Kriens wrote:
<snip>
> I must admit I feel I am walking on eggs ... and I am a bit surprised
> how few others tune in.
</snip>

there is a reason few others turn in, at this point, you have written,
and very well so, about 30 pages of responses.
It's just to hard keep up with long essays like that, not that I don't
want, I would love to, I just don't have the time.
If you keep your responses to a few short paragraphs, you might get more
input

Filip

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[jimmanico]

 > I just don't have the time. If you keep your responses to a few short
paragraphs, you might get more input

Filip, hey. May I ask when you think the HttpOnly patch will go live?
And Mark, I've spammed you about this as well - I'm running my own
custom branch eager to back back inline with the REAL Tomcat(tm).

Best,
Jim

> Peter Kriens wrote:
> <snip>
>> I must admit I feel I am walking on eggs ... and I am a bit surprised
>> how few others tune in.
> </snip>
>
> there is a reason few others turn in, at this point, you have written,
> and very well so, about 30 pages of responses.
> It's just to hard keep up with long essays like that, not that I don't
> want, I would love to, I just don't have the time.
> If you keep your responses to a few short paragraphs, you might get
> more input
>
> Filip
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

--
Jim Manico, Senior Application Security Engineer
jim.manico@... | jim@...
[phone] (work)
[phone] (cell)

Aspect Security™
Securing your applications at the source
http://www.aspectsecurity.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Costin Manolache-2]

Long thread, my summary:

1. Adding OSGI manifests to tomcat jars: there is interest, it will provide
benefits for people using tomcat in an OSGI environment. I don't think there
is any major controversy - it'll not affect any existing functionality. If
Henri or someone familiar with OSGi and interested can contribute the work -
it would be great.

2. Using OSGi for webapp loading/deployment: there is no direct benefit and
risks to alter the standard ( webapps are not allowed to share classes ) -
but it would be an interesting experiment for sandbox or some separate
module. It seems clear it won't provide 'gapless' deployment. If there are
volunteers - I'm curious to see how it works.

3. Using OSGi to modularize tomcat - i.e. to deploy things like connectors,
session managers, etc - Peter mentioned that it is possible to do it without
requiring each module to be started and using the activator to register
services. I believe there is interest in modularizing tomcat, and this seems
like one clear option - if it can be done non-intrusively ( i.e. it should
be in some separate package and should be possible to run tomcat as it is
now, without requiring OSGI ). That's the part I'm most interested in, I'll
probably play with this for tomcat-lite.

4. Using OSGi extensions to the servlet API ( HttpService ) - I'm not
convinced, but since it can be provided in a separate bundle I don't think
it's worth arguing about it.

5. OSGi defines a basic 'core' - i.e. bundles, manifests and advanced class
loader - and a number of extra services and APIs. Peter explained that the
core can be used without touching the 'extra', he obviously believes the
extras are valuable - I believe they are dangerous, but given that tomcat
already decided to use JMX and other equivalents - it would be a hard battle
to change. In any case - for any new dependency or API to be used in tomcat
I'm sure we can have a technical debate and decide what to do - since most
OSGi APIs depend on OSGi and we want OSGi to be optional ( i.e. to have
tomcat working with and without OSGI ) - it'll be an easy one :-)

IMO it is clear stability of the existing code (and community) is the major
concern, so 'walking on eggs' seems appropriate.

Costin

On Thu, May 1, 2008 at 1:24 AM, Peter Kriens <peter.kriens@...> wrote:

> Regarding 'dynamic register/unregister' - the servlet API defines one way
> > to
> > do this, i.e. war and the deployment. There is no standard API to
> > install/uninstall/start/stop a .war
> > - but HttpService is not that either. Runtime config changes (
> > adding/removing servlets
> > without web.xml changes and re-deployment ) is not specified, but it's a
> > whole different discussion
> > for the JSR people to solve, I'm pretty sure it'll be different from
> > HttpService.
> >
> Ok, but who is re-inventing here? :-)
>
>  Actually - JMX provides a lot of this dynamism.  Tomcat is using a lot of
> > JMX ( and hopefully will use more ), and provide very similar model with
> > OSGI services.
> >
> I think you underestimate what OSGi does and how closely the module layer,
> lifecycle layer,
> and service layer interact to make things work for the programmer. OSGi is
> really more than
> classloaders on steroids.
>
> However, referring to an earlier  discussion, I think the trick is not to
> become dependent on
> any one technology. I.e. the core Servlet and JSP engines should be POJO,
> and receive their
> connections from the outside. Be that JMX, OSGi, or whatever. This would
> make it trivial
> to provide the core functionality as a bean as well as an OSGi service. As
> long as the engines
> can handle the dynamism.
>
> The OSGi framework is about one idea: decoupling. All OSGi services are
> optional and none of the
> services is based on other services. The majority of bundles does not have
> to touch OSGi API. The
> recommendation has always been to configure your bundle in the bundle
> activator, but do not perform
> any user functionality there so you can reuse your implementations in
> other contexts.
>
> The OSGi functionality is limited to the management of the environment and
> how these decoupled units
> can leverage each other. Interestingly, the majority of open source
> projects have (re)invented this
> wheel over and over again, invariably not taking into account the
> important issues like versioning
> and strict modularity.
>
>  Webapps are quite similar with the bundles - it would be interesting to
> > see
> > if we could use OSGI classloader instead of ours, but I don't think
> > that's a real problem.
> >
> Well, you seem to have the desired functionality? Why switch? I would
> never update a project
> to use a new technology until it really is a significant improvement. I do
> not think the value
> proposition of the OSGi class loaders warrants the switch on its own?
>
>  People are looking for 'gapless' restart, i.e. users who have an active
> > session continuing to use the old version ( or some magic way to migrate
> > the session - but that's
> > likely impossible in most real cases, i.e. if code changes happen ).
> > OSGi alone can't solve
> > this.
> >
> Not on its own. However, the bundle model allows you to do a lot of
> interesting things here. In the
> Http Service model you know exactly the bundle that registered the servlet
> (through the use of an
> OSGi Service Factory). If the servlet is unregistered, I assume you can
> keep the session object around.
> the problem is, I am out of my league here, is that the session object
> contains classes
> to the old bundle. The trick is to convert these to the classes in the
> updated bundle. I guess you
> could use serialization (you know exactly the bundle where the new classes
> should come from) or a fancy
> reflection based mechanism, or use a mechanism that includes the bundle.
> Then again, this is not my area of expertise. However, I do believe that
> the bundle model and service model with their notification model do help
> allow you to do this
> without ending up in class cast exceptions and version mismatches.
>
> Of course there are nasty issues when classes in a session are reused by
> independent bundles. We have
> a similar (I think) problem with security. One bundle can use a different
> version of a permission class
> than another. We therefore convert the permission info to the correct
> version of the permission
> class when doing the check because we then know exactly what version that
> caller is using. You could
> do something similar with the session object. If an object is gotten from
> it, you can check if the servlet's
> bundle is compatible with the class in the session and automatically
> convert it to the appropriate class.
>
> Then again, I am not an expert here so this might not be the problem at
> all. I only know that the devil
> is in the details. :-(
>
>  A whole different class of users would like to see _less_ dynamism - i.e.
> > embedding tomcat as a jar and using simple APIs and no class loaders or
> > config files. In both
> > high-end servers and low-end embedding this is a very important use
> > case.
> >
> I agree, this is aligned with the POJO ideas. Separate the functionality
> in a set of completely disjoint
> functionalities. Then allow the customer to wire them into an OSGi based
> system, JMX based system,
> standalone server, etc. You can then always provide one or more of those
> instances. The trick is
> decoupling functional units from each other, with requires that things
> that are put together are
> highly cohesive. Anyway, no dynamism is the simple instance of dynamism
> ...
>
> I think the remainder of your mail is about this decomposition into
> functional units. I wholeheartedly
> agree that you want to minimize dependencies on OSGi API. In this phase I
> would need a whiteboard
> with people that have expertise in the different units to make the best
> decomposition. I will be at JavaOne next
> week so if people want to discuss the use of OSGi, feel free to contact me
> there.
>
>  ( well, we also have some social problems from time to time :-)
> >
> I must admit I feel I am walking on eggs ... and I am a bit surprised how
> few others tune in. If I am
> trespassing here, please let me know kindly :-)
>
> Kind regards,
>
>        Peter Kriens
>
>
>
>
>
>
>
>
> On 30 apr 2008, at 19:28, Costin Manolache wrote:
>
>  On Wed, Apr 30, 2008 at 1:00 AM, Peter Kriens <peter.kriens@...>
> > wrote:
> >
> >  Regarding HttpService - I don't think it's a good idea for tomcat.
> > >
> > > > One of the major problems with OSGI ( and we need to make sure we
> > > > don't
> > > > fall
> > > > in this trap ) is the re-invention of common APIs - logging, servlet
> > > > interfaces, etc.
> > > >
> > > >  As a bit of background. The logging and Http Service API are from
> > > 1999. At
> > > that time
> > > there was no dominant common logging API (neither in Java SE nor in
> > > open
> > > source),
> > > and the Http Service API is 100% based on the, at that time, standard
> > > Servlet API (it
> > > uses javax.servlet.http), it only provides an API to dynamically
> > > register
> > > and unregister
> > > servlets, which is still lacking in the current Servlet API.
> > >
> >
> >
> >
> > Regarding 'dynamic register/unregister' - the servlet API defines one
> > way to
> > do this, i.e. war and the
> > deployment. There is no standard API to install/uninstall/start/stop a
> > .war
> > - but HttpService
> > is not that either. Runtime config changes ( adding/removing servlets
> > without web.xml changes
> > and re-deployment ) is not specified, but it's a whole different
> > discussion
> > for the JSR people
> > to solve, I'm pretty sure it'll be different from HttpService.
> >
> >
> >
> > It would be quite inappropriate for tomcat to not use the standard
> >
> > > deployment/configuration mechanism for
> > > > servlets. So using or implementing any of the OSGI-re-defined
> > > > service
> > > > interfaces in
> > > > tomcat would be a hard sale IMO.
> > > >
> > > >
> > > Well, I do not see that this is an dichotomy. By nature of being the
> > > RI,
> > > you must follow the
> > > JSR. However, it would not be hard to provide the Http Service as an
> > > additional component. If
> > > Tomcat provides an API to dynamically register servlets, it would be
> > > trivial for someone
> > > to provide an OSGi compatibility bundle, just like people are doing it
> > > today.
> > > But it would be a nice service to get this from the horse's mouth. I
> > > am
> > > sure people are willing
> > > to provide this code.
> > >
> > >
> > A lot of people would like for tomcat to provide more jetty-like APIs
> > for
> > programmatic
> > configuration ( and in a way it is already possible - just not easy ).
> > As an API, HttpService is way off - in '99 and servlet 2.1 it may have
> > been
> > valuable.
> >
> > Let's keep HttpService for a different discussion :-)
> >
> >
> >
> >
> >
> >  In reality, this is a rather crude approach because in well designed
> > > systems the coupling between bundles
> > > is minimal. At this point in time, services usually start to look more
> > > attractive because they provide
> > > an API to allow dynamic updates without crudely stopping all bundles
> > > in
> > > the module dependency
> > > graph (which in non-service based systems, and especially
> > > require-bundle
> > > based systems tends
> > > to become the whole system). And a service is just a POJO that is
> > > registered under one or more interfaces. By allowing
> > > it to withdrawn at any moment in time, as well as registered by
> > > multiple
> > > independent parties, OSGi
> > > provides a good abstraction of this dynamism. And there is no Java
> > > counterpart for this.
> > >
> > >
> > Actually - JMX provides a lot of this dynamism.  Tomcat is using a lot
> > of
> > JMX ( and hopefully will use more ),
> > and provide very similar model with OSGI services.
> >
> >
> >
> >
> >
> >  they fell in love and the service layer was a major part of their
> > > infatuation. They
> > > realized very quickly how they could leverage the services as beans in
> > > their model and the
> > > advantages of dynamism without rebooting.
> > >
> >
> >
> > To clarify: updating webapps in tomcat without rebooting has been around
> > for
> > many years :-).
> > Webapps are quite similar with the bundles - it would be interesting to
> > see
> > if we could use
> > OSGI classloader instead of ours, but I don't think that's a real
> > problem.
> > People are looking for 'gapless' restart, i.e. users who have an active
> > session continuing to
> > use the old version ( or some magic way to migrate the session - but
> > that's
> > likely impossible
> > in most real cases, i.e. if code changes happen ). OSGi alone can't
> > solve
> > this.
> >
> > A whole different class of users would like to see _less_ dynamism -
> > i.e.
> > embedding tomcat
> > as a jar and using simple APIs and no class loaders or config files. In
> > both
> > high-end servers
> > and low-end embedding this is a very important use case.
> >
> >
> > The problem I would like to see solved in tomcat is breaking it up in
> > modules - i.e. separating
> > all the optional parts and having a way to ship a really minimal core
> > and a
> > good way to release and
> > deploy add-on bundles. OSGi may be a good way to do this.
> >
> > I think the requirements are:
> > - provide a way for different tomcat components and core to be packaged
> > as
> > OSGi bundles, using manifests
> > and maybe optional activators if it can't be avoided ( as long as tomcat
> > doesn't depend on it ). I think this
> > is an easy sell, I can't think of any good technical reasons not to do
> > it.
> >
> > - break tomcat release ( for 7.0 or 6.5 or whatever trunk will be
> > released
> > as ) into just core ( minimal servlet
> > impl ) and bundles
> >
> > - find a easy way for people to download/install all modules they want.
> >
> > - integrate this with the JMX layer and the manager servlet
> >
> > - all this without a hard dependency on OSGI - for people who want 'one
> > big
> > jar' for embedding
> > ( we don't want to force them to use osgi - or be dynamic - if they
> > don't
> > have a need for this)
> >
> > I think 'more dynamism' is a good thing - but so is 'less dynamism',
> >  both
> > are valid  goals and we  can and
> > should support both. ( right now we're somewhere in the middle -  don't
> > really support either one well enough )
> >
> >
> > Is there a way for a bundle to declare that it implements some services
> >
> > > using manifest or some config file, but without requiring it to be
> > > started
> > > to register
> > > services via BundleContext ? What I would like is to start tomcat, use
> > > server.xml
> > > to find what services are needed, and load the bundles providing the
> > > modules
> > > ( and only those ).
> > >
> > >  This is the use case of Declarative Services. The Service Component
> > Runtime
> >
> > > (SCR)
> > > will inspect the bundle when it gets installed and register any
> > > services
> > > that
> > > are declared and which have their dependencies on other services
> > > satisfied.
> > >
> >
> >
> > Any pointer to the docs ?
> >
> > If it only involves declarative stuff ( xml, manifest ) and is done at
> > install time - it may work
> > for us.
> >
> > Having a bundle start just to declare they provide a service (that won't
> > be
> > used) is a
> > non-starter.
> >
> > The current model in tomcat is to use Class.forName to load components
> > using
> > server.xml -
> > only the code that is used is started. I don't think it'll be acceptable
> > to
> > replace
> > server.xml or use OSGi for configuration ( in particular when JMX is a
> > better solution for
> > many config problems ).
> >
> >
> >
> >  The OSGi manifest does have a header Export-Service and Import-Service
> > > defined but because
> > > you can't give an guarantees about them without running the bundle, we
> > > decided to deprecate them.
> > >
> >
> >
> > Too bad :-)
> >
> >
> >
> > > However, how much do you want to be in the business of deployment?
> > > Tomcat
> > > is very often used
> > > as an application server but the web site tells me your core is a
> > > Servlet
> > > and
> > > JSP implementation.
> > >
> >
> >
> > Tomcat is in the business of deployment and configuration of WAR files -
> > that's a core part of the spec.
> >
> >
> >
> >  If you are going for strict modularity, wouldn't it make sense to
> > > distinguish
> > > between these core functions; Aren't the application management
> > > aspects
> > > secondary? I.e.
> > > allow your core JSP and Servlet engines to be used by others that
> > > provide
> > > the deployment
> > > aspects and allow your deployment code to be able to use other web
> > > service
> > > providers? How
> > > can you leverage existing deployment servers and management agents?
> > > For me
> > > the key thing about
> > > modularity is to do as little as possible and focus on core business.
> > >
> >
> >
> > Sure - the core business of tomcat is to deploy/configure/run WAR files
> > conforming to the spec.
> >
> >
> > The biggest problems ( IMO ) facing tomcat are:
> > - it needs more modularity in terms of its own extensions ( connectors,
> > auth, session managers, etc ).
> > - it needs to be easier to embed - both dynamically and as 'one big jar'
> > (
> > or jetty-style ).
> > For both OSGi could help.
> >
> > ( well, we also have some social problems from time to time :-)
> >
> > Costin
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>