Running XML Graphics products on OSGi

[Jeremias Maerki-2]

Over the past few months, I've started to get FOP, Batik and Commons
running in an OSGi environment. The first easy step is to just add the
necessary metadata to the manifest. However, that is not enough in the
case here. The problem: we're using the JAR Service Provider mechanism
from the JAR specification (META-INF/services directory in the JARs).

OSGi doesn't have a hierarchical class loader setup like traditional
Java applications which is why FOP, for example, may not see all
available plug-ins anymore if they are not compiled together into a
monster-JAR.

The solution was to build an abstraction layer above the direct access
to the META-INF/services files. In an OSGi-environment, a special
component (a so-called extender) will watch all available bundles (=JARs)
in the application. If it finds plug-ins it make them available despite
the class loader isolation. Well, that's simply the executive summary.
In the end, this is a replacement for the "Services" class in XML
Graphics Commons which we use today.

Anyway, I've published today an initial version of that abstraction
layer on my website [1]. I've started locally to change XML Graphics
Commons, so Commons can see the ImageConverter plug-ins provided by FOP
in an OSGi environment. With that alone I've already been able to run
FOP & Batik in OSGi. I'll have to do more of the same for all other
extension points. That means changes to all three products (including
Batik). Also, extension authors will have to make their plug-ins
OSGi-capable if they want their extensions to work in an OSGi
environment.

I'm going to wait a bit before proposing any patches. I first want to
get some feedback on the abstraction layer from the Felix community
where I've also posted the link. Felix might be one possible location
where this thing could be maintained. Since it also works completely
without OSGi, Apache Commons could be another option. That should be
sorted out in the following weeks. I just wanted to let everyone know
that this is something I would like to address in the near future.

If you want to know what OSGi is, take a quick look on the Wikipedia
page [2]. If you're an Eclipse user, you're already working on OSGi,
even though you may not even know.

[1] http://www.jeremias-maerki.ch/development/osgi/jar-services.html
[2] http://en.wikipedia.org/wiki/OSGi



Jeremias Maerki


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@...
For additional commands, e-mail: general-help@...

On Tue, Nov 03, 2009 at 05:18:05PM +0100, Jeremias Maerki wrote:

> Over the past few months, I've started to get FOP, Batik and Commons
> running in an OSGi environment. The first easy step is to just add the
> necessary metadata to the manifest. However, that is not enough in the
> case here. The problem: we're using the JAR Service Provider mechanism
> from the JAR specification (META-INF/services directory in the JARs).
>
> OSGi doesn't have a hierarchical class loader setup like traditional
> Java applications which is why FOP, for example, may not see all
> available plug-ins anymore if they are not compiled together into a
> monster-JAR.
>
> The solution was to build an abstraction layer above the direct access
> to the META-INF/services files. In an OSGi-environment, a special
> component (a so-called extender) will watch all available bundles (=JARs)
> in the application. If it finds plug-ins it make them available despite
> the class loader isolation. Well, that's simply the executive summary.
> In the end, this is a replacement for the "Services" class in XML
> Graphics Commons which we use today.

I am surprised that such a utility does not yet exist. Surely FOP is
not the only application that loads plug-ins. Eclipse is based on
plug-ins, but I suppose they solve the problem because you have to
install eclips plug-ins in known places.

Simon

--
Simon Pepping
home page: http://www.leverkruid.eu

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@...
For additional commands, e-mail: general-help@...

[Jeremias Maerki-2]

On 03.11.2009 21:51:53 Simon Pepping wrote:

> On Tue, Nov 03, 2009 at 05:18:05PM +0100, Jeremias Maerki wrote:
> > Over the past few months, I've started to get FOP, Batik and Commons
> > running in an OSGi environment. The first easy step is to just add the
> > necessary metadata to the manifest. However, that is not enough in the
> > case here. The problem: we're using the JAR Service Provider mechanism
> > from the JAR specification (META-INF/services directory in the JARs).
> >
> > OSGi doesn't have a hierarchical class loader setup like traditional
> > Java applications which is why FOP, for example, may not see all
> > available plug-ins anymore if they are not compiled together into a
> > monster-JAR.
> >
> > The solution was to build an abstraction layer above the direct access
> > to the META-INF/services files. In an OSGi-environment, a special
> > component (a so-called extender) will watch all available bundles (=JARs)
> > in the application. If it finds plug-ins it make them available despite
> > the class loader isolation. Well, that's simply the executive summary.
> > In the end, this is a replacement for the "Services" class in XML
> > Graphics Commons which we use today.
>
> I am surprised that such a utility does not yet exist. Surely FOP is
> not the only application that loads plug-ins. Eclipse is based on
> plug-ins, but I suppose they solve the problem because you have to
> install eclips plug-ins in known places.

Not sure I understanding what you're trying to say. OSGi is all about
services, plug-ins or whatever you call it. That's one of the great
things about it. Eclipse is based on OSGi but their way of doing
plug-ins is actually a bit special because Eclipse has a non-OSGi past
which I've read is still creaping around in today's design. In a "normal"
OSGi environment, you just install a new bundle and that bundle may
register a service with the OSGi service registry. And that service may
be a plug-in that can then be used by some other component.

On the other side (non-OSGi), we've solved the plug-in thing in XML
Graphics using the JAR service provider mechanism. Worked fine for us so
far as long as we're staying with the default hierarchical class loaders.

What was missing was something that makes the JAR service providers work
in an OSGi environment. There is a sub-specification in the OSGi
compendium that tries to address the discovery of XML parsers.
SAXParserFactory and DocumentBuilderFactory implementations are normally
also discovered through the JAR service provider mechanism in plain Java.
At the moment, most OSGi applications can just switch parsers by using
the "Endorsed Standards Override Mechanism" but not by deploying an XML
parser as normal OSGi bundle. Not very OSGi-like. But there's no OSGi
specification that defines how the JAR service provider mechanism in
general is mapped into OSGi. In Apache Felix, there have been some
efforts for discovering JAXB implementations, for example, but in these
cases, it's always about finding one implementation (i.e. the default
implementation). We actually need multiple implementations/plug-ins per
interface, not just one. And that simply wasn't covered (in OSGi land),
yet, as far as my research showed. That's why I built this "bridge". FOP
doesn't have to adopt OSGi (nor have a runtime dependency on any of its
spec JARs), but it can be made fully OSGi-compatible so someone can one
day just drop Barcode4J in a deployment hotfolder, Felix picks it up and
deploys it as a bundle and FOP can start immediately producing barcodes
despite the class loader barriers in OSGi. All at runtime. You can even
update the plug-in without restarting the application.

Not sure if I'm making sense.

Jeremias Maerki


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@...
For additional commands, e-mail: general-help@...

On Tue, Nov 03, 2009 at 11:13:44PM +0100, Jeremias Maerki wrote:
> On 03.11.2009 21:51:53 Simon Pepping wrote:
> > On Tue, Nov 03, 2009 at 05:18:05PM +0100, Jeremias Maerki wrote:
> Not sure I understanding what you're trying to say. OSGi is all about
> services, plug-ins or whatever you call it. That's one of the great
> things about it. Eclipse is based on OSGi but their way of doing

That is probably due to the fact that I do not know a thing about OSGi.

> Not sure if I'm making sense.

Sure, thanks.

Simon

--
Simon Pepping
home page: http://www.leverkruid.eu

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@...
For additional commands, e-mail: general-help@...

[Max Berger]

Jeremias,

a late reply, but hopefully still valid:

- If you use OSGi, you will either need to implement a simplified
version for detecting plug-ins, or introduce additional dependencies
(apache felix). Both would increase the size of xmlgraphics-commons, so
should be carefully considered.

- at least until fop 1.0 plug-ins written using the service approach
should still be loaded.

- I like the OSGi approach as it is currently widely used.

As an idea: If you submit your code to apache felix maybe they could
produce some kind of "felix lite" version, which would be the minimal
osgi environment needed to replace the current "Service" class? This
would remove my first point above

Max


Jeremias Maerki schrieb:

> Over the past few months, I've started to get FOP, Batik and Commons
> running in an OSGi environment. The first easy step is to just add the
> necessary metadata to the manifest. However, that is not enough in the
> case here. The problem: we're using the JAR Service Provider mechanism
> from the JAR specification (META-INF/services directory in the JARs).
>
> OSGi doesn't have a hierarchical class loader setup like traditional
> Java applications which is why FOP, for example, may not see all
> available plug-ins anymore if they are not compiled together into a
> monster-JAR.
>
> The solution was to build an abstraction layer above the direct access
> to the META-INF/services files. In an OSGi-environment, a special
> component (a so-called extender) will watch all available bundles (=JARs)
> in the application. If it finds plug-ins it make them available despite
> the class loader isolation. Well, that's simply the executive summary.
> In the end, this is a replacement for the "Services" class in XML
> Graphics Commons which we use today.
>
> Anyway, I've published today an initial version of that abstraction
> layer on my website [1]. I've started locally to change XML Graphics
> Commons, so Commons can see the ImageConverter plug-ins provided by FOP
> in an OSGi environment. With that alone I've already been able to run
> FOP & Batik in OSGi. I'll have to do more of the same for all other
> extension points. That means changes to all three products (including
> Batik). Also, extension authors will have to make their plug-ins
> OSGi-capable if they want their extensions to work in an OSGi
> environment.
>
> I'm going to wait a bit before proposing any patches. I first want to
> get some feedback on the abstraction layer from the Felix community
> where I've also posted the link. Felix might be one possible location
> where this thing could be maintained. Since it also works completely
> without OSGi, Apache Commons could be another option. That should be
> sorted out in the following weeks. I just wanted to let everyone know
> that this is something I would like to address in the near future.
>
> If you want to know what OSGi is, take a quick look on the Wikipedia
> page [2]. If you're an Eclipse user, you're already working on OSGi,
> even though you may not even know.
>
> [1] http://www.jeremias-maerki.ch/development/osgi/jar-services.html
> [2] http://en.wikipedia.org/wiki/OSGi
>
>
>
> Jeremias Maerki

--
http://max.berger.name/
OpenPGP ID: C93C5700 Fpr: AB6638CE472A499B3959 ADA2F989A2E5C93C5700

[Jeremias Maerki-2]

Hi Max

On 12.11.2009 11:12:02 Max Berger wrote:
> Jeremias,
>
> a late reply, but hopefully still valid:
>
> - If you use OSGi, you will either need to implement a simplified
> version for detecting plug-ins, or introduce additional dependencies
> (apache felix). Both would increase the size of xmlgraphics-commons, so
> should be carefully considered.

Yes, there will be an increase of classes necessary to handle this (-1
Services.java in XGC, and 1+ JAR of 28KB (15 classes)), but no, there
won't be a dependency on Apache Felix or any other OSGi specific class
in XGC. That's what my abstraction layer is all about: shielding
FOP/Batik from OSGi-specifics but enabling them to run in an OSGi
environment.

> - at least until fop 1.0 plug-ins written using the service approach
> should still be loaded.

My abstraction layer is designed to be a complete replacement of the
Services class but in terms of plug-in handling almost nothing changes.
The META-INF/service is still used as before. Instead of getting an
iterator with all plug-in classes you set up a simple listener which
gets notified if a plug-in becomes available or goes away. I don't
propose to suddenly use OSGi services at all. This whole thing will
still work completely without a single OSGi or Felix class in the
classpath for the big majority of users who still don't use OSGi.

> - I like the OSGi approach as it is currently widely used.

Not as widely as you might think. Otherwise, people would have come
screaming for OSGi metadata in FOP a long time ago. But yeah, OSGi is
very cool and adoption is spreading fast.

> As an idea: If you submit your code to apache felix maybe they could
> produce some kind of "felix lite" version, which would be the minimal
> osgi environment needed to replace the current "Service" class? This
> would remove my first point above

I think with my above comments I've already addressed that, but without
the need for a "Felix lite".

> Max
>
>
> Jeremias Maerki schrieb:
> > Over the past few months, I've started to get FOP, Batik and Commons
> > running in an OSGi environment. The first easy step is to just add the
> > necessary metadata to the manifest. However, that is not enough in the
> > case here. The problem: we're using the JAR Service Provider mechanism
> > from the JAR specification (META-INF/services directory in the JARs).
> >
> > OSGi doesn't have a hierarchical class loader setup like traditional
> > Java applications which is why FOP, for example, may not see all
> > available plug-ins anymore if they are not compiled together into a
> > monster-JAR.
> >
> > The solution was to build an abstraction layer above the direct access
> > to the META-INF/services files. In an OSGi-environment, a special
> > component (a so-called extender) will watch all available bundles (=JARs)
> > in the application. If it finds plug-ins it make them available despite
> > the class loader isolation. Well, that's simply the executive summary.
> > In the end, this is a replacement for the "Services" class in XML
> > Graphics Commons which we use today.
> >
> > Anyway, I've published today an initial version of that abstraction
> > layer on my website [1]. I've started locally to change XML Graphics
> > Commons, so Commons can see the ImageConverter plug-ins provided by FOP
> > in an OSGi environment. With that alone I've already been able to run
> > FOP & Batik in OSGi. I'll have to do more of the same for all other
> > extension points. That means changes to all three products (including
> > Batik). Also, extension authors will have to make their plug-ins
> > OSGi-capable if they want their extensions to work in an OSGi
> > environment.
> >
> > I'm going to wait a bit before proposing any patches. I first want to
> > get some feedback on the abstraction layer from the Felix community
> > where I've also posted the link. Felix might be one possible location
> > where this thing could be maintained. Since it also works completely
> > without OSGi, Apache Commons could be another option. That should be
> > sorted out in the following weeks. I just wanted to let everyone know
> > that this is something I would like to address in the near future.
> >
> > If you want to know what OSGi is, take a quick look on the Wikipedia
> > page [2]. If you're an Eclipse user, you're already working on OSGi,
> > even though you may not even know.
> >
> > [1] http://www.jeremias-maerki.ch/development/osgi/jar-services.html
> > [2] http://en.wikipedia.org/wiki/OSGi
> >
> >
> >
> > Jeremias Maerki
>
>
>
> --
> http://max.berger.name/
> OpenPGP ID: C93C5700 Fpr: AB6638CE472A499B3959 ADA2F989A2E5C93C5700
>

Jeremias Maerki


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@...
For additional commands, e-mail: general-help@...