[jira] Created: (HTTPCLIENT-841) potential memory leak when using ThreadSafeClientConnManager

potential memory leak when using ThreadSafeClientConnManager
------------------------------------------------------------

                 Key: HTTPCLIENT-841
                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
             Project: HttpComponents HttpClient
          Issue Type: Bug
          Components: HttpClient
    Affects Versions: 4.0 Beta 2
         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
            Reporter: Ted Slusser


When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.

Here is a diff from 4.0-beta2


--- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
+++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
@@ -50,7 +50,7 @@
     protected final ReferenceQueue<?> refQueue;
 
     /** The handler for the references found. */
-    protected final RefQueueHandler refHandler;
+    protected RefQueueHandler refHandler;
 
 
     /**
@@ -112,6 +112,8 @@
             this.workerThread = null; // indicate shutdown
             wt.interrupt();
         }
+
+        refHandler = null;
     }
 
 


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Oleg Kalnichevski updated HTTPCLIENT-841:
-----------------------------------------

    Fix Version/s: 4.0 Beta 3

Bizarre. Anyone has an idea why this should matter?

Oleg

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699645#action_12699645 ]

Sebb commented on HTTPCLIENT-841:
---------------------------------

This may be completely off-base, but:

Is it possible that the RefQueueWorker shutdown() method was called directly, rather than using the appropriate pool shutdown() method(s)?

[Maybe the RefQueueWorker shutdown() method should be package scope?]

Might be worth adding some debug code to check whether or not the refHandler resources have all been released.
Not sure where to put this though, as RefQueueWorker.shutdown() is called first.

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699683#action_12699683 ]

Oleg Kalnichevski commented on HTTPCLIENT-841:
----------------------------------------------

This code was originally written by Roland and I suspect he never managed to get around to finishing it completely. I am a bit uneasy about touching component no one seems to have good knowledge of and making changes no one seems to be completely sure about.

I am leaning towards just disabling garbage collection of connections for the 4.0 release and revisiting this problem in the course of 4.1 development.

Any objections to that?

Oleg

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699686#action_12699686 ]

Sam Berlin commented on HTTPCLIENT-841:
---------------------------------------

I have a little familiarity with it. I'll take a look -- I think the garbage collection closing is a useful thing to keep, if possible.  

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Ortwin Glück]

Oleg Kalnichevski (JIRA) wrote:
> I am leaning towards just disabling garbage collection of connections for the
> 4.0 release and revisiting this problem in the course of 4.1 development.
>
> Any objections to that?
>
> Oleg

Anyway this connection GC is a hackish solution approach for people who don't
properly manage connections. It is not something that should be necessary in a
properly coded application. In the end it will byte your ass, because it abuses
the garbage collector as a general resource manager, which it simply isn't. GC
only kicks in when a heap generation fills up, but not when the system is
running low on file handles. It's a big mistake to use the GC for other things
than memory management.

Cheers,

Ortwin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699701#action_12699701 ]

Sebb commented on HTTPCLIENT-841:
---------------------------------

The implementation is currently difficult to make thread-safe, so removing it would be good.

If redesigned, it would be good to remove/rework the enableConnectionGC() method, which can only be called immediately after creating the pool.
The method creates additional instance objects (refQueue and refWorker); if these could be optionally created at construction time they could be made final to improve thread-safety. There would still need to be a way to start the RefQueueWorker thread after the AbstractConnPool was created, but this could be done by the concrete subclass constructor (as is done now).

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699790#action_12699790 ]

Oleg Kalnichevski commented on HTTPCLIENT-841:
----------------------------------------------

Sam,

By all of means please do take a stab at it. However, personally I would prefer to be on the cautious side and exclude this feature for the 4.0 release anyway.

Oleg

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699794#action_12699794 ]

Sebb commented on HTTPCLIENT-841:
---------------------------------

If the feature was made optional - e.g. via a parameter as hinted at in the code - then the default could be to not include it.

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Oleg Kalnichevski updated HTTPCLIENT-841:
-----------------------------------------

    Attachment: HTTPCLIENT-841.patch

Folks

I took a look at this issue and this is what I have to say. With Roland gone I do not see a way of coming up with a reliable fix for the problem without removing / disabling the connection garbage collection code. Moreover, I can't help thinking this feature causes more harm than good. It basically makes ThreadSafeClientConnManager, well, not thread safe. Personally I am prepared to go as far as breaking the API compatibility to have a better peace of mind about reliability of ThreadSafeClientConnManager and to simplify a fairly complex code, which all this GC magic makes even more complex.

I'll once again try to get in touch with Google folks to find out how they feel about possibility of breaking API / binary compatibility with Android.

I am attaching a patch that completely rips out garbage collection of connections.

Oleg

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>         Attachments: HTTPCLIENT-841.patch
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Oleg Kalnichevski updated HTTPCLIENT-841:
-----------------------------------------

    Attachment: HTTPCLIENT-841-depr.patch

Similar patch but retaining 100% API compatibility. If I hear no objections, and no one steps in to fix the connection GC code, I'll commit this patch in a few days and close the issue.

Oleg

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>         Attachments: HTTPCLIENT-841-depr.patch, HTTPCLIENT-841.patch
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Oleg Kalnichevski resolved HTTPCLIENT-841.
------------------------------------------

    Resolution: Fixed

Patch checked in. Please re-test

Oleg

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Beta 3
>
>         Attachments: HTTPCLIENT-841-depr.patch, HTTPCLIENT-841.patch
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754148#action_12754148 ]

Olivier Lamy commented on HTTPCLIENT-841:
-----------------------------------------

So personnaly I have some issues with currently.
Is it really fixed ?
I will attached a jvm error pid.

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Final
>
>         Attachments: HTTPCLIENT-841-depr.patch, HTTPCLIENT-841.patch
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olivier Lamy updated HTTPCLIENT-841:
------------------------------------

    Attachment: hs_err_pid21886.log

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Final
>
>         Attachments: hs_err_pid21886.log, HTTPCLIENT-841-depr.patch, HTTPCLIENT-841.patch
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754163#action_12754163 ]

Oleg Kalnichevski commented on HTTPCLIENT-841:
----------------------------------------------

Olivier,

Yes, I am pretty certain HTTPCLIENT-841as been fixed. I have taken a cursory look at the jvm dump and so far I see no evidence of a memory leak or HttpClient being the cause of the JVM crash. JVM process died unexpectedly, that is for sure, but the reasons appears to be an internal JVM bug

---
#  Internal Error (sharedRuntime.cpp:461), pid=21886, tid=20
#  Error: guarantee(false,"missing exception handler")
---

> potential memory leak when using ThreadSafeClientConnManager
> ------------------------------------------------------------
>
>                 Key: HTTPCLIENT-841
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-841
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Leopard 10.5.6, Java 1.5.0_16, Jetty 6.1.7
>            Reporter: Ted Slusser
>             Fix For: 4.0 Final
>
>         Attachments: hs_err_pid21886.log, HTTPCLIENT-841-depr.patch, HTTPCLIENT-841.patch
>
>
> When using ThreadSafeClientConnManager and developing with Jetty using auto-redeploy feature eventually I run into a PermGen out of memory exception.  I investigated with YourKit 8.0.6 and found a class loader circular reference in RefQueueWorker.  Not really sure what I was doing I made the refQueueHandler non-final and nulled it in the shutdown method of RedQueueWorker.  I don't seem to have the problem any longer with circular class loader references.
> Here is a diff from 4.0-beta2
> --- httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(revision 763223)
> +++ httpclient/src/main/java/org/apache/http/impl/conn/tsccm/RefQueueWorker.jav(working copy)
> @@ -50,7 +50,7 @@
>      protected final ReferenceQueue<?> refQueue;
>  
>      /** The handler for the references found. */
> -    protected final RefQueueHandler refHandler;
> +    protected RefQueueHandler refHandler;
>  
>  
>      /**
> @@ -112,6 +112,8 @@
>              this.workerThread = null; // indicate shutdown
>              wt.interrupt();
>          }
> +
> +        refHandler = null;
>      }
>  
>  

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...