Re: FW: Janino and Code Cache OutOfMemoryError

View: New views
2 Messages — Rating Filter:   Alert me  

Parent Message unknown Re: FW: Janino and Code Cache OutOfMemoryError

by Paul Nolan :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I should also add, when I look at what references org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0, I get pages and pages of classes that reference it, including many static references to it from "system" classes. This does not make a lot of sense since the classes that would be required to compile the code passed in to the ClassBodyEvaluator are not that many. For example, here is a reference to the ByteArrayClassLoader:

Static reference from net.sf.cglib.core.ReflectUtils.defaultLoader (from class net.sf.cglib.core.ReflectUtils) :

--> java.net.URLClassLoader@0x18030720 (63 bytes) (field classes:)
--> java.util.Vector@0x18079b60 (24 bytes) (field elementData:)
--> [Ljava.lang.Object;@0x191ff2a8 (40968 bytes) (Element 769 of [Ljava.lang.Object;@0x191ff2a8:)
--> class <mycompany>.cuf.datasource.portal.PortalDataSource (84 bytes) (static field _connectionManager:)
--> <mycompany>.jaf.services.database.ConnectionManagerBase@0x18030090 (184 bytes) (field _checkerThread:)
--> java.lang.Thread@0x18ac30d8 (104 bytes) (field group:)
--> java.lang.ThreadGroup@0x18030030 (43 bytes) (field groups:)
--> [Ljava.lang.ThreadGroup;@0x1a8024a0 (40 bytes) (Element 0 of [Ljava.lang.ThreadGroup;@0x1a8024a0:)
--> java.lang.ThreadGroup@0x180798f8 (43 bytes) (field threads:)
--> [Ljava.lang.Thread;@0x1b176678 (40 bytes) (Element 2 of [Ljava.lang.Thread;@0x1b176678:)
--> java.lang.Thread@0x1a92b240 (104 bytes) (field threadLocals:)
--> java.lang.ThreadLocal$ThreadLocalMap@0x1ab3b138 (20 bytes) (field table:)
--> [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68 (264 bytes) (Element 12 of [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68:)
--> java.lang.ThreadLocal$ThreadLocalMap$Entry@0x1af03218 (28 bytes) (field value:)
--> com.<mycompany>.ro.plugin.rules.RuleValidation$CommonBody@0x1af031c8 (16 bytes) (field cbe:)
--> org.codehaus.janino.ClassBodyEvaluator@0x1af031e8 (48 bytes) (field result:)
--> org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0 (54 bytes)

here is another:

Static reference from org.apache.commons.configuration.ConfigurationUtils.class$org$apache$commons$configuration$ConfigurationUtils (from class org.apache.commons.configuration.ConfigurationUtils) :

--> class org.apache.commons.configuration.ConfigurationUtils (84 bytes) (??:)
--> java.net.URLClassLoader@0x18030720 (63 bytes) (field classes:)
--> java.util.Vector@0x18079b60 (24 bytes) (field elementData:)
--> [Ljava.lang.Object;@0x191ff2a8 (40968 bytes) (Element 769 of [Ljava.lang.Object;@0x191ff2a8:)
--> class <mycompany>.cuf.datasource.portal.PortalDataSource (84 bytes) (static field _connectionManager:)
--> <mycompany>.jaf.services.database.ConnectionManagerBase@0x18030090 (184 bytes) (field _checkerThread:)
--> java.lang.Thread@0x18ac30d8 (104 bytes) (field group:)
--> java.lang.ThreadGroup@0x18030030 (43 bytes) (field groups:)
--> [Ljava.lang.ThreadGroup;@0x1a8024a0 (40 bytes) (Element 0 of [Ljava.lang.ThreadGroup;@0x1a8024a0:)
--> java.lang.ThreadGroup@0x180798f8 (43 bytes) (field threads:)
--> [Ljava.lang.Thread;@0x1b176678 (40 bytes) (Element 2 of [Ljava.lang.Thread;@0x1b176678:)
--> java.lang.Thread@0x1a92b240 (104 bytes) (field threadLocals:)
--> java.lang.ThreadLocal$ThreadLocalMap@0x1ab3b138 (20 bytes) (field table:)
--> [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68 (264 bytes) (Element 12 of [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68:)
--> java.lang.ThreadLocal$ThreadLocalMap$Entry@0x1af03218 (28 bytes) (field value:)
--> com.<mycompany>.ro.plugin.rules.RuleValidation$CommonBody@0x1af031c8 (16 bytes) (field cbe:)
--> org.codehaus.janino.ClassBodyEvaluator@0x1af031e8 (48 bytes) (field result:)
--> org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0 (54 bytes)

thanks,

- Paul.

On Tue, Oct 27, 2009 at 9:07 AM, Paul Nolan <Paul.Nolan@...> wrote:

 

 

From: Paul Nolan [mailto:paul.nolan@...]
Sent: 27 October 2009 08:58
To: user@...
Subject: [janino-user] Janino and Code Cache OutOfMemoryError

 

Hi,

I have a ituation where I am running some code inside our home grown application server. This code uses Janino to generate code on the fly using the ClassBodyEvaluator and, to a lesser extend, the SimpleCompiler. This code is not executed in any way - we compile the code to "test" rules input by users into our system to make sure that the rules are correct.

First I should say that permGen space is *not* filling up, just the code cache. I am seeing a situation where the code cache is filling up. Now, my understanding of this is that the reason the code cache is filling up is due to some reference to the class compiled by ClassBodyEvaluator is still hanging around in the system. However, my confusion is this: If this is the case then surely the permGen space should alse fill up (or possibly it does so at a much smaller pace than the code cache since it stores meta data only?).

I have passed a custom class loader into the ClassBodyEvaluator to see if this solves the issue but it does not e.g.

            ClassBodyEvaluator cbe = new ClassBodyEvaluator();
            cbe.setParentClassLoader(RVClassLoaderFactory.getRVClassLoader());
            cbe.cook(code);

In my testing code, I nullify my custom class loader when I am finished with it but the code cache still grows and does not get garbage collected. Using JDK 6.

I have used jmap and jhat to try and see what is referencing my class, the following information is from a jmap dump:

      1. ClassLoader for my generated class: org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0 (54 bytes) (note: not my custom classLoader I pass in above).

      2. In my investiagations using jmap, the finger consistently points at org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0, even though I pass in a custom class loader above when I am generating the code.

Now, it is well possible that my understanding of how Janino uses classLoaders is limited or how classLoaders work, so if so apologies.

My general question is can someone enlighten me as to the best way to ensure that my code cache does not fill up? Is it as simple as calling Thread.currentThread().setContextClassLoader() ?

regards,

- Paul.
          

   



Re: FW: Janino and Code Cache OutOfMemoryError

by Matt Fowles :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Paul~

I am not entirely sure about your situation, but I would guess that this is from instances of the generated classes surviving longer then you expect.  Every object in Java holds a reference to its Class and every Class holds an reference to its ClassLoader, so if your code instantiates the Class that you are generating, then the code objects and the ClassLoader will survive as long as that instance does.

If you are not instantiating the classes at all, I would examine the static and thread local variables that are referenced in your traces below to ensure that they are not squirreling away Class or ClassLoader references.

Matt





On Tue, Oct 27, 2009 at 5:14 AM, Paul Nolan <paul.nolan@...> wrote:
I should also add, when I look at what references org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0, I get pages and pages of classes that reference it, including many static references to it from "system" classes. This does not make a lot of sense since the classes that would be required to compile the code passed in to the ClassBodyEvaluator are not that many. For example, here is a reference to the ByteArrayClassLoader:

Static reference from net.sf.cglib.core.ReflectUtils.defaultLoader (from class net.sf.cglib.core.ReflectUtils) :

--> java.net.URLClassLoader@0x18030720 (63 bytes) (field classes:)
--> java.util.Vector@0x18079b60 (24 bytes) (field elementData:)
--> [Ljava.lang.Object;@0x191ff2a8 (40968 bytes) (Element 769 of [Ljava.lang.Object;@0x191ff2a8:)
--> class <mycompany>.cuf.datasource.portal.PortalDataSource (84 bytes) (static field _connectionManager:)
--> <mycompany>.jaf.services.database.ConnectionManagerBase@0x18030090 (184 bytes) (field _checkerThread:)
--> java.lang.Thread@0x18ac30d8 (104 bytes) (field group:)
--> java.lang.ThreadGroup@0x18030030 (43 bytes) (field groups:)
--> [Ljava.lang.ThreadGroup;@0x1a8024a0 (40 bytes) (Element 0 of [Ljava.lang.ThreadGroup;@0x1a8024a0:)
--> java.lang.ThreadGroup@0x180798f8 (43 bytes) (field threads:)
--> [Ljava.lang.Thread;@0x1b176678 (40 bytes) (Element 2 of [Ljava.lang.Thread;@0x1b176678:)
--> java.lang.Thread@0x1a92b240 (104 bytes) (field threadLocals:)
--> java.lang.ThreadLocal$ThreadLocalMap@0x1ab3b138 (20 bytes) (field table:)
--> [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68 (264 bytes) (Element 12 of [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68:)
--> java.lang.ThreadLocal$ThreadLocalMap$Entry@0x1af03218 (28 bytes) (field value:)
--> com.<mycompany>.ro.plugin.rules.RuleValidation$CommonBody@0x1af031c8 (16 bytes) (field cbe:)
--> org.codehaus.janino.ClassBodyEvaluator@0x1af031e8 (48 bytes) (field result:)
--> org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0 (54 bytes)

here is another:

Static reference from org.apache.commons.configuration.ConfigurationUtils.class$org$apache$commons$configuration$ConfigurationUtils (from class org.apache.commons.configuration.ConfigurationUtils) :

--> class org.apache.commons.configuration.ConfigurationUtils (84 bytes) (??:)
--> java.net.URLClassLoader@0x18030720 (63 bytes) (field classes:)
--> java.util.Vector@0x18079b60 (24 bytes) (field elementData:)
--> [Ljava.lang.Object;@0x191ff2a8 (40968 bytes) (Element 769 of [Ljava.lang.Object;@0x191ff2a8:)
--> class <mycompany>.cuf.datasource.portal.PortalDataSource (84 bytes) (static field _connectionManager:)
--> <mycompany>.jaf.services.database.ConnectionManagerBase@0x18030090 (184 bytes) (field _checkerThread:)
--> java.lang.Thread@0x18ac30d8 (104 bytes) (field group:)
--> java.lang.ThreadGroup@0x18030030 (43 bytes) (field groups:)
--> [Ljava.lang.ThreadGroup;@0x1a8024a0 (40 bytes) (Element 0 of [Ljava.lang.ThreadGroup;@0x1a8024a0:)
--> java.lang.ThreadGroup@0x180798f8 (43 bytes) (field threads:)
--> [Ljava.lang.Thread;@0x1b176678 (40 bytes) (Element 2 of [Ljava.lang.Thread;@0x1b176678:)
--> java.lang.Thread@0x1a92b240 (104 bytes) (field threadLocals:)
--> java.lang.ThreadLocal$ThreadLocalMap@0x1ab3b138 (20 bytes) (field table:)
--> [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68 (264 bytes) (Element 12 of [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@0x1add1f68:)
--> java.lang.ThreadLocal$ThreadLocalMap$Entry@0x1af03218 (28 bytes) (field value:)
--> com.<mycompany>.ro.plugin.rules.RuleValidation$CommonBody@0x1af031c8 (16 bytes) (field cbe:)
--> org.codehaus.janino.ClassBodyEvaluator@0x1af031e8 (48 bytes) (field result:)
--> org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0 (54 bytes)

thanks,

- Paul.

On Tue, Oct 27, 2009 at 9:07 AM, Paul Nolan <Paul.Nolan@...> wrote:

 

 

From: Paul Nolan [mailto:paul.nolan@...]
Sent: 27 October 2009 08:58
To: user@...
Subject: [janino-user] Janino and Code Cache OutOfMemoryError

 

Hi,

I have a ituation where I am running some code inside our home grown application server. This code uses Janino to generate code on the fly using the ClassBodyEvaluator and, to a lesser extend, the SimpleCompiler. This code is not executed in any way - we compile the code to "test" rules input by users into our system to make sure that the rules are correct.

First I should say that permGen space is *not* filling up, just the code cache. I am seeing a situation where the code cache is filling up. Now, my understanding of this is that the reason the code cache is filling up is due to some reference to the class compiled by ClassBodyEvaluator is still hanging around in the system. However, my confusion is this: If this is the case then surely the permGen space should alse fill up (or possibly it does so at a much smaller pace than the code cache since it stores meta data only?).

I have passed a custom class loader into the ClassBodyEvaluator to see if this solves the issue but it does not e.g.

            ClassBodyEvaluator cbe = new ClassBodyEvaluator();
            cbe.setParentClassLoader(RVClassLoaderFactory.getRVClassLoader());
            cbe.cook(code);

In my testing code, I nullify my custom class loader when I am finished with it but the code cache still grows and does not get garbage collected. Using JDK 6.

I have used jmap and jhat to try and see what is referencing my class, the following information is from a jmap dump:

      1. ClassLoader for my generated class: org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0 (54 bytes) (note: not my custom classLoader I pass in above).

      2. In my investiagations using jmap, the finger consistently points at org.codehaus.janino.ByteArrayClassLoader@0x1af0f9f0, even though I pass in a custom class loader above when I am generating the code.

Now, it is well possible that my understanding of how Janino uses classLoaders is limited or how classLoaders work, so if so apologies.

My general question is can someone enlighten me as to the best way to ensure that my code cache does not fill up? Is it as simple as calling Thread.currentThread().setContextClassLoader() ?

regards,

- Paul.