ClassLoader leaks?

[Kathey Marsden]

I got this analysis from a user and am not quite sure what
to make of it yet, but thought I would forward it to the
list to see if anyone has opinions or it makes more sense
to them than it does to me at this point.  Here's the note.

"I've  found a few app ClassLoader leaks in Derby.

...The first one looks like Derby created a long-running
thread and copying the context class loader.  To fix, the
context class loader should be saved/set/restored around the
creation of the new thread so that it copies some benign class
loader instead (e.g., null or getClass().getClassLoader()):

 0x42278e58 java/lang/Thread@302e302e
  [truncating at running thread LEAK]

Object:  0x42278e58 java/lang/Thread@302e302e
Children:
 0x42278ee0 java/lang/String@303f303f
 0x4226e558 java/lang/ThreadGroup@6f2e6f2e
 0x42278e40
org/apache/derby/impl/services/monitor/AntiGC@603a603a
 0x419cfac0

The second is another long running thread.  The same applies:

 0x426fe7a0 java/lang/Thread@19901990
  [truncating at running thread LEAK]

Object:  0x426fe7a0 java/lang/Thread@19901990
Parents:
 0x4226e5a8 [Ljava/lang/Thread;@6f386f38
 0x426fe548
org/apache/derby/iapi/services/context/ContextManager@19421942
Children:
 0x426fe838 java/lang/String@19a319a3
 0x4226e558 java/lang/ThreadGroup@6f2e6f2e
 0x426fe4f8
org/apache/derby/impl/services/daemon/BasicDaemon@19381938
 0x419cfac0

The third is a TimerThread owneed , which is created when a
Timer is created.  The same applies:

 0x425ac538 java/util/Timer$TimerImpl@6b8a6b8a
  [truncating at running thread LEAK]

Object:  0x425ac538 java/util/Timer$TimerImpl@6b8a6b8a
Parents:
 0x41faaf58 [Ljava/lang/Thread;@3c583c58

Object:  0x425ac510 java/util/Timer@6b856b85
Parents:
 0x425ac500
org/apache/derby/impl/services/timer/SingletonTimerFactory@56e25
6e2


Thanks for any input. I will follow up with the user and
see if I can get some more details and translate into
something we can file in Jira.

Kathey

[Kathey Marsden]

Kathey Marsden wrote:

> I got this analysis from a user and am not quite sure what to make of
> it yet, but thought I would forward it to the
> list to see if anyone has opinions or it makes more sense to them than
> it does to me at this point.  Here's the note.
>
> "I've  found a few app ClassLoader leaks in Derby.
>
> ...The first one looks like Derby created a long-running thread and
> copying the context class loader.  To fix, the context class loader
> should be saved/set/restored around the creation of the new thread so
> that it copies some benign class loader instead (e.g., null or
> getClass().getClassLoader()):
>
> 0x42278e58 java/lang/Thread@302e302e
>  [truncating at running thread LEAK]
>
> Object:  0x42278e58 java/lang/Thread@302e302e
> Children:
> 0x42278ee0 java/lang/String@303f303f
> 0x4226e558 java/lang/ThreadGroup@6f2e6f2e
> 0x42278e40 org/apache/derby/impl/services/monitor/AntiGC@603a603a
> 0x419cfac0
> The second is another long running thread.  The same applies:
>
> 0x426fe7a0 java/lang/Thread@19901990
>  [truncating at running thread LEAK]
>
> Object:  0x426fe7a0 java/lang/Thread@19901990
> Parents:
> 0x4226e5a8 [Ljava/lang/Thread;@6f386f38
> 0x426fe548 org/apache/derby/iapi/services/context/ContextManager@19421942
> Children:
> 0x426fe838 java/lang/String@19a319a3
> 0x4226e558 java/lang/ThreadGroup@6f2e6f2e
> 0x426fe4f8 org/apache/derby/impl/services/daemon/BasicDaemon@19381938
> 0x419cfac0
> The third is a TimerThread owneed , which is created when a Timer is
> created.  The same applies:
>
> 0x425ac538 java/util/Timer$TimerImpl@6b8a6b8a
>  [truncating at running thread LEAK]
>
> Object:  0x425ac538 java/util/Timer$TimerImpl@6b8a6b8a
> Parents:
> 0x41faaf58 [Ljava/lang/Thread;@3c583c58
>
> Object:  0x425ac510 java/util/Timer@6b856b85
> Parents:
> 0x425ac500
> org/apache/derby/impl/services/timer/SingletonTimerFactory@56e25
> 6e2
>
>
> Thanks for any input. I will follow up with the user and see if I can
> get some more details and translate into something we can file in Jira.
>

I got some more information on this issue.  The user modified a simple

derby program to "show the leak".  This work's only with IBM jdk's.  
Here is the comment that went with the program:

"I create a new BJKCL and set it as the context class loader.  Then, we
interact with Derby and clean up the context class loader.  Finally, run
a GC which should allow the temporary BJKCL to be cleaned up, and then
generate a javacore/heapdump.  In an app server  scenario, that would
translate as "BJKCL is the app CL", and the "interact with Derby" is an
application doing a JNDI DataSource lookup for the first time.  Of
course, we never want to shut down Derby in the app server, so I had to
remove that bit to keep things alive."

Thanks again for any input.  I'm still not totally clear on what is
going on here.


Kathey



import java.sql.*;

public class SimpleDerbyProgramShowLeak {

    public static void main(String[] args) throws Exception{
        // setup
        Thread.currentThread().setContextClassLoader(new BJKCL());

        // use derby
        Class.forName("org.apache.derby.jdbc.EmbeddedDriver").newInstance();
        ((BJKCL)Thread.currentThread().getContextClassLoader()).defineClass().newInstance();

        // cleanup
        Thread.currentThread().setContextClassLoader(null);

        // generate dumps
        System.gc();
        com.ibm.jvm.Dump.HeapDump();
        com.ibm.jvm.Dump.JavaDump();
    }

    public SimpleDerbyProgramShowLeak() throws Exception {
        Connection conn = DriverManager.getConnection("jdbc:derby:wombat;create=true");
        Statement s = conn.createStatement();
        ResultSet rs = s.executeQuery("SELECT COUNT(*) from SYS.SYSTABLES");
        rs.next();
        System.out.println("count of systables = " + rs.getString(1));

/*
        // Shutdown derby. We expect an exception on shutdown
        try {
            DriverManager.getConnection("jdbc:derby:;shutdown=true");
        } catch (SQLException se) {

        }
*/
    }
}

class BJKCL extends ClassLoader {
    Class defineClass() throws Exception {
        byte[] b = new byte[4096];
        return defineClass(b, 0, new java.io.FileInputStream("SimpleDerbyProgramShowLeak.class").read(b));
    }
}

[Kathey Marsden]

A little more info. A couple of references and a summary.
http://opensource.atlassian.com/confluence/spring/pages/viewpage.action?pageId=2669
http://blogs.sun.com/fkieviet/entry/classloader_leaks_the_dreaded_java

A brief summary: Starting an enterprise application creates a new
ClassLoader.  Stopping it should cause the CL to go away.  If some
component fails to release a reference, then repeatedly restarting an
application will eventually exhaust system memory.  In this case, I
suspect these are long-running threads that only ever leak one CL.  
However, it is still desirable to clean these references up because (1)
they clutter heap dumps and make analysis of other problems more
difficult and (2) if many components leaked different CLs (different
code paths as the app is repeatedly restarted) memory consumption issues
can still occur.