Seeking authoritative answer re using Log4J with v6

[Jonathan Ross]

I'd like to use Log4J at the server level, but following the instructions at apache.org doesn't seeem to work with 6.0.18.

1)  Rebuilding tomcat-juli.jar using extras.xml doesn't include an adapter for Log4J.  The build file lists org/apache/juli/logging/impl/Jdk14Logger.class, but not org/apache/juli/logging/impl/Log4JLogger.class, and indeed if you turn on Commons Logging diagnostics, you can see it failing to find Log4JLogger and then finding Jdk14Logger in the JAR (and then trying to fail back to JDK logging.)

2)  If I fix the build file to include Log4JLogger, it still fails; the Commons Logging diagnostics say the class is "missing dependencies", which occurs when the instantiation throws a NoClassDefFoundError.  Not surprising; if log4j.jar is placed in catalina.home/lib per instructions, it's not on the system classpath, which is used by bootstrap.jar.

3)  Because the build of tomcat-juli.jar doesn't include the JULI default log handlers, the fallback to JDK logging fails in either case, with a ClassNotFoundException on org.apache.juli.FileHandler.

I can't move log4j.jar to the system classpath, because then it will be visible to all webapps, and I will have to use a custom Log4J RepositorySelector (presumably with a context classloader-keyed hash map et cetera, ugh.)

Another option is to re-root all of Log4J under org.apache.juli and duplicate it in tomcat-juli.jar.  Double ugh, no thanks.

I would be OK with using JDK logging just during server bootstrap, if I could get Log4J for the rest of the server (via the Common classpath.)

Has anyone made this work?

Thanks,
Jonathan

[Caldarale, Charles R]

> From: Jonathan Ross [mailto:jonross@...]
> Subject: Seeking authoritative answer re using Log4J with v6
>
> if log4j.jar is placed in catalina.home/lib per instructions,
> it's not on the system classpath, which is used by bootstrap.jar.

A recent posting by Mark T indicates the doc might be in error; someone writing a custom log handler had to place it on Tomcat's startup classpath, so the same might very well be true for log4j.  Some experimentation is in order, and you may have just done that.

> I can't move log4j.jar to the system classpath, because then it
> will be visible to all webapps

It was already visible to all webapps when the jar was in Tomcat's lib directory, so moving it should not matter.  The local copies in WEB-INF/lib *should* be found before the system classloader is requested.  Note that in order to get individual use of log4j by a webapp, each must have its own log4j.jar and log4j.properties.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Jonathan Ross]

> > I can't move log4j.jar to the system classpath, because then it
> > will be visible to all webapps

> It was already visible to all webapps when the jar was in Tomcat's lib directory, so moving it
> should not matter.  The local copies in WEB-INF/lib *should* be found before the system
> classloader is requested.  Note that in order to get individual use of log4j by a webapp,
> each must have its own log4j.jar and log4j.properties.

According to the classloader howto, the order is: system classloader, webapp classloader, common classloader.  So I'll get a global RepositorySelector if I move log4j.jar to the startup classpath.

[Caldarale, Charles R]

> From: Jonathan Ross [mailto:jonross@...]
> Subject: RE: Seeking authoritative answer re using Log4J with v6
>
> According to the
> http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html
> classloader howto , the order is: system classloader, webapp
> classloader, common classloader.

The code does not match the doc, nor observed behavior.  The WebappClassLoader does have a delegate field that can be used to force a search via the parent and system classloaders before WEB-INF/lib and WEB-INF/classes, but the value of that field is normally false.  It can be set by the delegate attribute of a <Loader> element nested inside a <Context> element, but it's unlikely you're doing that.  When delegate is false (the default), the WEB-INF directories are searched before system or common.

The WebappClassLoader also has a provision for forcing delegation for certain packages, but that package list is empty in the current implementation.  It could be filled in by a subclass (but there are none in Tomcat).

All of the above is based on 6.0.20.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Mark Thomas]

Caldarale, Charles R wrote:
>> From: Jonathan Ross [mailto:jonross@...]
>> Subject: Seeking authoritative answer re using Log4J with v6
>>
>> if log4j.jar is placed in catalina.home/lib per instructions,
>> it's not on the system classpath, which is used by bootstrap.jar.
>
> A recent posting by Mark T indicates the doc might be in error; someone writing a custom log handler had to place it on Tomcat's startup classpath, so the same might very well be true for log4j.  Some experimentation is in order, and you may have just done that.

I have checked recently (with 6.0.18) and it worked as expected and documented.
I'll check 6.0.20 and post the results.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Mark Thomas]

Mark Thomas wrote:

> Caldarale, Charles R wrote:
>>> From: Jonathan Ross [mailto:jonross@...]
>>> Subject: Seeking authoritative answer re using Log4J with v6
>>>
>>> if log4j.jar is placed in catalina.home/lib per instructions,
>>> it's not on the system classpath, which is used by bootstrap.jar.
>> A recent posting by Mark T indicates the doc might be in error; someone writing a custom log handler had to place it on Tomcat's startup classpath, so the same might very well be true for log4j.  Some experimentation is in order, and you may have just done that.
>
> I have checked recently (with 6.0.18) and it worked as expected and documented.
> I'll check 6.0.20 and post the results.

6.0.20 works for me as well. It looks very much like your build environment is
broken. Rather than building the extras package yourself, just download it. This
works for me with 6.0.18 and 6.0.20 as documented.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Caldarale, Charles R]

> From: Mark Thomas [mailto:markt@...]
> Subject: Re: Seeking authoritative answer re using Log4J with v6
>
> It looks very much like your build environment is broken.

I just duplicated Jonathan's findings on my Vista box.  There are several things broken, but the build environment doesn't appear to be one of them.

1) The extras.xml script places Log4JLogger.class in tomcat-juli-adapters.jar, but not tomcat-juli.jar, thus preventing Tomcat from using log4j internally.
 
2) Several classes are duplicated in the extras versions of tomcat-juli.jar and tomcat-juli-adapters.jar:
  org/apache/juli/logging/impl/Jdk14Logger.class
  org/apache/juli/logging/impl/LogFactoryImpl$1.class
  org/apache/juli/logging/impl/LogFactoryImpl$2.class
  org/apache/juli/logging/impl/LogFactoryImpl$3.class
  org/apache/juli/logging/impl/NoOpLog.class
  org/apache/juli/logging/impl/SimpleLog.class
  org/apache/juli/logging/impl/SimpleLog$1.class
Most of this is likely intentional, but each jar contains different additional classes in the org.apache.juli.logging.impl package, which strikes me as odd.

3) org/apache/juli/logging/impl/LogFactoryImpl.class is missing from tomcat-juli-adapters.jar, despite its internal classes being there.

The above is true both for jars downloaded from bin/extras and for ones built via extras.xml.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Caldarale, Charles R]

> From: Mark Thomas [mailto:markt@...]
> Subject: Re: Seeking authoritative answer re using Log4J with v6
>
> I have checked recently (with 6.0.18) and it worked as
> expected and documented.

So why was it necessary for Suresh to place his log handler on Tomcat's startup classpath?

Or to put it another way, how can logging initialization find log4j.jar in Tomcat's lib directory, but not find Suresh's custom handler in the same location?

Jonathan's intent is to have Tomcat use log4j for all its internal logging, not just webapp-related logging.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Mark Thomas]

Caldarale, Charles R wrote:
>> From: Mark Thomas [mailto:markt@...]
>> Subject: Re: Seeking authoritative answer re using Log4J with v6
>>
>> It looks very much like your build environment is broken.
>
> I just duplicated Jonathan's findings on my Vista box.  There are several things broken, but the build environment doesn't appear to be one of them.
>
> 1) The extras.xml script places Log4JLogger.class in tomcat-juli-adapters.jar, but not tomcat-juli.jar, thus preventing Tomcat from using log4j internally.

I haven't looked at what gets placed where, but following the instructions on
the logging page to replace j.u.l with log4j works for me. The only change I
made was to set the org.apache logger level to INFO rather than debug and I get
a log4j version of the standard Tomcat startup.

I'm testing this on a mac but I have had this working in windows in the past too.

Are you sure you followed all of the steps on the logging page with a clean
Tomcat install? I don't understand why this is working for me but not for you.

If you follow the instructions on the logging page, modify the last line of the
log4j.properties file to:

log4j.logger.org.apache=INFO, R

What do you get in CATALINA_BASE\logs\tomcat.log when you call catalina.bat start?

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Mark Thomas]

Caldarale, Charles R wrote:
>> From: Mark Thomas [mailto:markt@...]
>> Subject: Re: Seeking authoritative answer re using Log4J with v6
>>
>> I have checked recently (with 6.0.18) and it worked as
>> expected and documented.
>
> So why was it necessary for Suresh to place his log handler on Tomcat's startup classpath?

I suspect, but haven't confirmed, that this was because juli reads its
properties file on initialisation, wanted to instantiate the handler and hadn't
reached the point where the common class loader had been set up.

> Or to put it another way, how can logging initialization find log4j.jar in Tomcat's lib directory, but not find Suresh's custom handler in the same location?

Of the top of my head, no idea. Again, my suspicion is that it is related to
what gets instantiated when compared to when the common class loader is configured.

> Jonathan's intent is to have Tomcat use log4j for all its internal logging, not just webapp-related logging.

Yes, I am aware of that.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Caldarale, Charles R]

> From: Mark Thomas [mailto:markt@...]
> Subject: Re: Seeking authoritative answer re using Log4J with v6
>
> Are you sure you followed all of the steps on the logging page with a
> clean Tomcat install?

I will try again from scratch, and report back.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Jonathan Ross]

Thanks, this worked.  I downloaded tomcat-juli.jar from http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.18/bin/extras/, replaced my build with it, and now have log4j at the server level.  (I still need to very the classloader order prefers log4j.jar in a webapp.)

Mark Thomas-18 wrote:

Mark Thomas wrote:
> I have checked recently (with 6.0.18) and it worked as expected and documented.
> I'll check 6.0.20 and post the results.

6.0.20 works for me as well. It looks very much like your build environment is
broken. Rather than building the extras package yourself, just download it. This
works for me with 6.0.18 and 6.0.20 as documented.

Mark

[Caldarale, Charles R]

> From: Caldarale, Charles R
> Subject: RE: Seeking authoritative answer re using Log4J with v6
>
> > Are you sure you followed all of the steps on the logging page with a
> > clean Tomcat install?
>
> I will try again from scratch, and report back.

Did a clean install of Win 7, JDK 6u14, ant 1.7.1, and Tomcat 6.0.20, and downloaded the extras jars rather than building them.  Configured as documented on the logging page - and it all worked as described there, with tomcat.1 and tomcat.log being generated (using DEBUG logging).

Still trying to figure out the differences between that and what was done on the Vista system...

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Mike Frohme]

This also worked correctly (as documented) for me with 6.0.20 - with the build of extras and log4j off the SVN trunk (patches for certain property handling due out in 1.2.16)

--
Mike

-----Original Message-----
From: Mark Thomas [mailto:markt@...]
Sent: Friday, June 26, 2009 4:01 PM
To: Tomcat Users List
Subject: Re: Seeking authoritative answer re using Log4J with v6

Mark Thomas wrote:

> Caldarale, Charles R wrote:
>>> From: Jonathan Ross [mailto:jonross@...]
>>> Subject: Seeking authoritative answer re using Log4J with v6
>>>
>>> if log4j.jar is placed in catalina.home/lib per instructions,
>>> it's not on the system classpath, which is used by bootstrap.jar.
>> A recent posting by Mark T indicates the doc might be in error; someone writing a custom log handler had to place it on Tomcat's startup classpath, so the same might very well be true for log4j.  Some experimentation is in order, and you may have just done that.
>
> I have checked recently (with 6.0.18) and it worked as expected and documented.
> I'll check 6.0.20 and post the results.

6.0.20 works for me as well. It looks very much like your build environment is
broken. Rather than building the extras package yourself, just download it. This
works for me with 6.0.18 and 6.0.20 as documented.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Jonathan Ross]

Sorry for the delay, had a few vacation days.

I ran some tests to verify Log4J config at webapp vs server level and had these additional observations:

• The log4j.properties file in CATALINA_HOME/lib is still used for initialization unless overridden in the default locations (log4j.properties or log4j.xml) by the webapp; using an alternate location via e.g. Spring Log4jContextListener will result in two LogManagers configured against the same files.
• JDK logging is still initialized by Tomcat's default logging.properties, resulting in several zero-length files in CATALINA_BASE/logs.

So overall that's three changes to the Tomcat documentation for Log4J:

• Use tomcat-juli.jar and tomcat-juli-adapters.jar from the archives; don't build them yourself (until the build script is fixed)
• Webapp must configure Log4j from the default locations (log4j.xml or log4j.properties at top-level in the classpath) or else supply an empty file at one of these locations to suppress the one on the server classpath
• CATALINA_BASE/conf/logging.properties should be deleted or renamed.

Jonathan Ross wrote:

Thanks, this worked. I downloaded tomcat-juli.jar from http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.18/bin/extras/, replaced my build with it, and now have log4j at the server level. (I still need to very the classloader order prefers log4j.jar in a webapp.)

Mark Thomas-18 wrote:

Mark Thomas wrote: > I have checked recently (with 6.0.18) and it worked as expected and documented. > I'll check 6.0.20 and post the results. 6.0.20 works for me as well. It looks very much like your build environment is broken. Rather than building the extras package yourself, just download it. This works for me with 6.0.18 and 6.0.20 as documented. Mark