[c3-monitoring] Overview of Servlet-Services

[Dariusz Łuksza]

Hi all,

Now I'm moving to next task, with would be implementation of
Servlet-Services overview.

I'm thtinking that it would be very nice if we can have separate MBean
instance for each Servlet-Service that would be contacting data only
for that single Servlet-Service. I'm pretty sure that I've spot
somewhere how to done it using Spring ... but now I can't remembe how
to done it ... and even worst, I can't remember how and where I found
it :|

Maybe you can help me ?

The main idea is to have separate configuration for monitoring beans
that don't interfere in other parts of Cocoon 3. That "magic
configuration" should made new instance of ServiceServletMonitoring
MBean for each initialized Service-Servlet.

Have you Any ideas how to achieve that ?

--
Best regards

Blog: http://luksza.org
LinkedIn: http://www.linkedin.com/in/dariuszluksza

[Reinhard Pötz-2]

Dariusz Łuksza wrote:

> Hi all,
>
> Now I'm moving to next task, with would be implementation of
> Servlet-Services overview.
>
> I'm thtinking that it would be very nice if we can have separate MBean
> instance for each Servlet-Service that would be contacting data only
> for that single Servlet-Service. I'm pretty sure that I've spot
> somewhere how to done it using Spring ... but now I can't remembe how
> to done it ... and even worst, I can't remember how and where I found
> it :|
>
> Maybe you can help me ?

You should be able to collect all Servlet-Services by using the Spring
Configurator's bean-map:
http://cocoon.apache.org/subprojects/configuration/1.0/spring-configurator/1.0/1400_1_1.html

> The main idea is to have separate configuration for monitoring beans
> that don't interfere in other parts of Cocoon 3. That "magic
> configuration" should made new instance of ServiceServletMonitoring
> MBean for each initialized Service-Servlet.

yes, I agree

> Have you Any ideas how to achieve that ?

Every servlet-service implements the javax.servlet.Servlet interface.
You can create a bean-map that filters by that type.

--
Reinhard Pötz                           Managing Director, {Indoqa} GmbH
                         http://www.indoqa.com/en/people/reinhard.poetz/

Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member                  reinhard@...
________________________________________________________________________

[Andreas Pinter]

Hello there,

We are going to send our first major patch. We are aware of the fact,
that our communication to the community was rather small up until now.

So I'll try to explain our current concept:

cocoon-profiling is a tool to determine how long the execution of a
request/pipeline took. It will show the specific times of every
component, so it should be very useful for debugging complex
applications.  We do not profile any memory usage.

This subproject is split into two components: cocoon-profiling is the
"backend" which does all the profiling work; cocoon-profiling-firebug is a
extension for firebug (a debugging/dev tool for firefox) to display the
information to the user.  It is written in JavaScript.

Our backend is based on (Spring-)AOP to keep the impact on the cocoon code
low. The next paragraphs give a more detailed introduction on how cocoon
profiling works.

We'll start with the explanation of our three
aspects, which try to intercept every "interesting" cocoon method
invocation.
These focus on the main parts of cocoon:
- ServletProfilingApsect           captures every invocation of Servlet
- SitemapNodeProfilingAspect       same for every SitemapNode
- PipelineComponentProfilingAspect and for every PipelineComponent

Every Aspect has an instance of an InvocationDispatcher.  The aspect
just forwards the method invocation information to its dispatcher.

At this point I need to say something about the distribution in the
background: As you can see in the application context
(cocoon-profiling-component.xml) each dispatcher owns a so-called
default profiler. A dispacher can hold a bunch of (specific / custom)
profilers, which can be added by the user.  This means that you can e.g.
add a profiler for HttpServlet which captures more information than the
default Servlet profiler.

Back to the process: Our Dispatcher now chooses the most specific
profiler and calls its "before" method. After this the catched method is
processed (so we let Cocoon3 do its work) and afterwards the Dispatcher
calls the profiler.after() method or the profiler.exception() -
depending on the result of the method invocation.  We pass a new
instance of ProfilingData to these methods.  A profiler can add
information about what happened to this instance.

At this point of the process we have collected all the data, but not
saved it.  So inside the "after" method, we pass the ProfilingData to
the ProfilingDataManager. This class sorts the ProfilingData depending
on their invocation level (based on the time / place of the profiler
call) and builds a tree of ProfilingData objects, which is maintained by
the profiler.

When the DataManger notices that it just got to the last ProfilingData
it passes the information to the ProfilingDataHolder, which is just a
simple map of String (which is the profiling id) and data.

At this point of the process, the cocoon work is done and the user sees
his result in the browser. This result hasn't changed in any way from a
call without profiling (it may have took a little bit longer, of course
  ;)  except from an additional HTTPResponseHeader information. We insert
the aforementioned profiling id in the special X-Cocoon-Profiling-ID
header.

This ID is used by the Firebug plugin to start a new request to a
specific profiling-pipeline.  The pipeline gets the ProfilingData
information using the DataHolder and generates XML with a SAX generator
and a xml serializer. If you don't like firebug you could process this
XML in your own tool to display profiling information.

So how does the plugin work?  First of all it is important to know, that
our Firebug plugin attaches itself to the Net-Tab of Firebug. (If you
don't know what I'm speaking about, try to use the 1.4X Version of
Firebug - the X is no typo) Via the Net-Tab we are informed about a
finished response, which we scan for the above mentioned
X-Cocoon-Profiling-ID. If we find this ID we take it and do our own
"hidden" request for ProfilingData. The response of this request is
processed by the Firebug plugin and builds a TreeView out of the
returned XML file.

The next steps for the whole process will be to produce a XMLSchema file
for the presented XML data, so we can make the plugin work better,
because it's not just a stupid XML-TreeView-Explorer any more.  On the
other hand we try to move the Cocoon-Profiling-Tab into the NetTab to
keep track of different Requests/Responses.

If you have any questions or suggestions, please feel free to share.  We
really need some feedback to see if we are going in the right direction.

Greetings,
Andreas

[Dariusz Łuksza]

On Mon, Jun 1, 2009 at 11:54 AM, Reinhard Pötz<reinhard@...> wrote:
>
> You should be able to collect all Servlet-Services by using the Spring
> Configurator's bean-map:
> http://cocoon.apache.org/subprojects/configuration/1.0/spring-configurator/1.0/1400_1_1.html
>

It is exactly what I was looking for, thanks :)

>
>> The main idea is to have separate configuration for monitoring beans
>> that don't interfere in other parts of Cocoon 3. That "magic
>> configuration" should made new instance of ServiceServletMonitoring
>> MBean for each initialized Service-Servlet.
>
> yes, I agree
>
>> Have you Any ideas how to achieve that ?
>
> Every servlet-service implements the javax.servlet.Servlet interface.
> You can create a bean-map that filters by that type.
>

Whole week I try to get somehow list of available Service-Servlets and
mount points for them. For now I think that there aren't any
possibility of getting that information using existing API.

IMHO simplest way of getting list of avialiable Servlet-Services would
be parse sitemap.xmap but I don't know how to distiguish with pipeline
use s-s and with isn't.

It would be werry nice if somebody could gave me some hits how move on
with this ;)

--
Best regards

Blog: http://luksza.org
LinkedIn: http://www.linkedin.com/in/dariuszluksza

[Reinhard Pötz-2]

Dariusz Łuksza wrote:

> On Mon, Jun 1, 2009 at 11:54 AM, Reinhard Pötz<reinhard@...> wrote:
>> You should be able to collect all Servlet-Services by using the Spring
>> Configurator's bean-map:
>> http://cocoon.apache.org/subprojects/configuration/1.0/spring-configurator/1.0/1400_1_1.html
>>
>
> It is exactly what I was looking for, thanks :)
>
>>> The main idea is to have separate configuration for monitoring beans
>>> that don't interfere in other parts of Cocoon 3. That "magic
>>> configuration" should made new instance of ServiceServletMonitoring
>>> MBean for each initialized Service-Servlet.
>> yes, I agree
>>
>>> Have you Any ideas how to achieve that ?
>> Every servlet-service implements the javax.servlet.Servlet interface.
>> You can create a bean-map that filters by that type.
>>
>
> Whole week I try to get somehow list of available Service-Servlets and
> mount points for them. For now I think that there aren't any
> possibility of getting that information using existing API.
>
> IMHO simplest way of getting list of avialiable Servlet-Services would
> be parse sitemap.xmap but I don't know how to distiguish with pipeline
> use s-s and with isn't.
>
> It would be werry nice if somebody could gave me some hits how move on
> with this ;)

IIRC each servlet-service has a special servlet context
o.a.c.servletservice.ServletServiceContext (cocoon-servlet-service-impl
module) which contains all necessary information (connections,
mount-points) but it doesn't expose them. If you can cast the
ServletContext to ServletServiceContext, you can safely assume that the
bean is a servlet-service.

I suggest that you expose the necessary methods, or just for now, use a
private accessor.

HTH

--
Reinhard Pötz                           Managing Director, {Indoqa} GmbH
                         http://www.indoqa.com/en/people/reinhard.poetz/

Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member                  reinhard@...
________________________________________________________________________

[Dariusz Łuksza]

On Tue, Jun 9, 2009 at 9:55 AM, Reinhard Pötz<reinhard@...> wrote:

>
> IIRC each servlet-service has a special servlet context
> o.a.c.servletservice.ServletServiceContext (cocoon-servlet-service-impl
> module) which contains all necessary information (connections,
> mount-points) but it doesn't expose them. If you can cast the
> ServletContext to ServletServiceContext, you can safely assume that the
> bean is a servlet-service.
>
> I suggest that you expose the necessary methods, or just for now, use a
> private accessor.
>
> HTH
>

Yes, that helps me a lot. I've all ready created task in jira
(COCOON3-38) and attached patches for this milestone.

Generally, when I was witting my GSoC proposal I had in my mind
another set of features, at that time "connections" means for me
opened http connections not internal Servlet-Service connections. But
now when I know better ideas of Servlet-Services and Cocoon I know
that getting that feature done would be very hard to achieve and
requires lots of changes in many places. So for now I implement only
small set of features ...

--
Best regards

Blog: http://luksza.org
LinkedIn: http://www.linkedin.com/in/dariuszluksza

[Reinhard Pötz-2]

Dariusz Łuksza wrote:

> On Tue, Jun 9, 2009 at 9:55 AM, Reinhard Pötz<reinhard@...> wrote:
>> IIRC each servlet-service has a special servlet context
>> o.a.c.servletservice.ServletServiceContext (cocoon-servlet-service-impl
>> module) which contains all necessary information (connections,
>> mount-points) but it doesn't expose them. If you can cast the
>> ServletContext to ServletServiceContext, you can safely assume that the
>> bean is a servlet-service.
>>
>> I suggest that you expose the necessary methods, or just for now, use a
>> private accessor.
>>
>> HTH
>>
>
> Yes, that helps me a lot. I've all ready created task in jira
> (COCOON3-38) and attached patches for this milestone.
>
> Generally, when I was witting my GSoC proposal I had in my mind
> another set of features, at that time "connections" means for me
> opened http connections not internal Servlet-Service connections.

Dariusz,

yes, that's out of scope of the GSoC project and I think that
Tomcat/Jetty already provide such information.

> But
> now when I know better ideas of Servlet-Services and Cocoon I know
> that getting that feature done would be very hard to achieve and
> requires lots of changes in many places. So for now I implement only
> small set of features ...

... that already look good!

I only have some (minor) suggestions:

1) I would prefer having all servlet services as nodes in the tree of
mbeans (similar to what Jetty does with servlets) and

2) also expose the servlet parameters.

3) show the short name of a connection additionally to the fully
qualified bean name

Thanks!

--
Reinhard Pötz                           Managing Director, {Indoqa} GmbH
                         http://www.indoqa.com/en/people/reinhard.poetz/

Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member                  reinhard@...
________________________________________________________________________

[Dariusz Łuksza]

On Mon, Jun 15, 2009 at 10:50 PM, Reinhard Pötz <reinhard@...> wrote:

Dariusz Łuksza wrote:
> On Tue, Jun 9, 2009 at 9:55 AM, Reinhard Pötz<reinhard@...> wrote:
>> IIRC each servlet-service has a special servlet context
>> o.a.c.servletservice.ServletServiceContext (cocoon-servlet-service-impl
>> module) which contains all necessary information (connections,
>> mount-points) but it doesn't expose them. If you can cast the
>> ServletContext to ServletServiceContext, you can safely assume that the
>> bean is a servlet-service.
>>
>> I suggest that you expose the necessary methods, or just for now, use a
>> private accessor.
>>
>> HTH
>>
>
> Yes, that helps me a lot. I've all ready created task in jira
> (COCOON3-38) and attached patches for this milestone.
>
> Generally, when I was witting my GSoC proposal I had in my mind
> another set of features, at that time "connections" means for me
> opened http connections not internal Servlet-Service connections.

Dariusz,

yes, that's out of scope of the GSoC project and I think that
Tomcat/Jetty already provide such information.

> But
> now when I know better ideas of Servlet-Services and Cocoon I know
> that getting that feature done would be very hard to achieve and
> requires lots of changes in many places. So for now I implement only
> small set of features ...

... that already look good!

I only have some (minor) suggestions:

1) I would prefer having all servlet services as nodes in the tree of
mbeans (similar to what Jetty does with servlets) and

2) also expose the servlet parameters.

3) show the short name of a connection additionally to the fully
qualified bean name

Suggestion number 2 and 3 are very simple, but your first suggestion, for now sounds quite difficult for me. I think that I should one again look at Spring JMX documentation and maybe I'll find simple solution for that task.

I think that up to next Monday or Tuesday I'll provide patch that will be covering all that features.

PS. Sorry for responding that late, last weak was quite exhausting for me at the university. In next weak I'll have exams, so I'll reduce amount of time that I spent on GSoC to minimum.

--

Best regards

Blog: http://luksza.org
LinkedIn: http://www.linkedin.com/in/dariuszluksza

[Dariusz Łuksza]

On Mon, Jun 15, 2009 at 10:50 PM, Reinhard Pötz <reinhard@...> wrote:

Dariusz Łuksza wrote:
> On Tue, Jun 9, 2009 at 9:55 AM, Reinhard Pötz<reinhard@...> wrote:
>> IIRC each servlet-service has a special servlet context
>> o.a.c.servletservice.ServletServiceContext (cocoon-servlet-service-impl
>> module) which contains all necessary information (connections,
>> mount-points) but it doesn't expose them. If you can cast the
>> ServletContext to ServletServiceContext, you can safely assume that the
>> bean is a servlet-service.
>>
>> I suggest that you expose the necessary methods, or just for now, use a
>> private accessor.
>>
>> HTH
>>
>
> Yes, that helps me a lot. I've all ready created task in jira
> (COCOON3-38) and attached patches for this milestone.
>
> Generally, when I was witting my GSoC proposal I had in my mind
> another set of features, at that time "connections" means for me
> opened http connections not internal Servlet-Service connections.

Dariusz,

yes, that's out of scope of the GSoC project and I think that
Tomcat/Jetty already provide such information.

> But
> now when I know better ideas of Servlet-Services and Cocoon I know
> that getting that feature done would be very hard to achieve and
> requires lots of changes in many places. So for now I implement only
> small set of features ...

... that already look good!

I only have some (minor) suggestions:

1) I would prefer having all servlet services as nodes in the tree of
mbeans (similar to what Jetty does with servlets) and

2) also expose the servlet parameters.

3) show the short name of a connection additionally to the fully
qualified bean name

 

I've all ready attached new patch[1] to COCOON3-38 jira issue that covers that 3 new features.

 

[1] https://issues.apache.org/jira/secure/attachment/12411359/servlet-services-monitor1.patch

 

--

Best regards

Blog: http://luksza.org
LinkedIn: http://www.linkedin.com/in/dariuszluksza