|
»
Memory Leak?
|
View:
New views
2 Messages
—
Rating Filter:
Alert me
|
 |
Memory Leak?
by athomson
::
Rate this Message:
Mike,
Seeing a memory leak in the latest jcifs 1.3.12 or it sure looks like one. This is from a long running process -- typically after about 10 hours it will error with an out of memory condition. System has 64GB memory java -Xmx1024m jcifs.smb.client.ssnLimit=1 "Exception in thread "Transport414026" java.lang.OutOfMemoryError: Java heap space" I happened to grab a heap dump about 11 hours into the latest run, ironically it ran out of memory shortly afterwards. What it looks like is that the connections are never released. More about this after some of the statistical information from the jhat tool. Listed below is a partial list containing a count of the all instances in the dump, this is just the top of the list. After 11 hours of running there are 304,000+ instances of various jcifs objects. Notice that the one of the largest consumers is a LinkedList. These objects account for 739 MB of "retained" memory in this heap dump. Instance Counts for All Classes (including platform) 1519162 instances of class [Ljava.lang.Object; 1236988 instances of class java.lang.Object 1219633 instances of class java.util.LinkedList$Entry 630506 instances of class java.lang.String 630493 instances of class [C 609819 instances of class java.util.LinkedList 607371 instances of class [S 313855 instances of class [B 312116 instances of class java.net.Inet4Address 310583 instances of class java.net.Socket 310583 instances of class java.net.SocksSocketImpl 305744 instances of class java.lang.ref.Finalizer 305253 instances of class [Ljava.util.HashMap$Entry; 305246 instances of class java.util.HashMap 304934 instances of class java.util.Vector 304925 instances of class jcifs.smb.SmbComBlankResponse 304908 instances of class jcifs.smb.SmbTransport 304908 instances of class jcifs.smb.SmbTransport$ServerData 304907 instances of class jcifs.UniAddress 304907 instances of class jcifs.smb.SmbSession 304907 instances of class jcifs.smb.SmbTree 302970 instances of class java.net.ConnectException 302695 instances of class jcifs.util.transport.TransportException An outline of the code is here, it's not really relevant, more for reference. It just chains an SmbFile opening a server/share, then a directory, and finally a file, which is written to using SmbFileOutputStream. Everything is in try {} catch {} finally {} blocks to make sure nothing escapes :-). This is most important for the stream to insure its closed no matter what. SmbFile targetServer = new SmbFile(machinePath, auth); do some stuff Smbfile targetDirectory = new SmbFile(smbTargetServer, directory); do more stuff SmbFile smbTargetFile = new SmbFile(smbTargetDir, fileName); do even more stuff SmbFileOutputStream o =null; try { o = new SmbFileOutputStream(smbTargetFile); write(); catch { } finally { try { o.close(); catch (Exception ex) { } } --- Going back to the heap dump, one of the biggest consumers is a LinkedList. Drilling into the heap, the entries are from SmbConstants and the list contains a reference called CONNECTIONS. Looking at the source file indicates that there is indeed a LinkedList called CONNECTIONS. jcifs.smb.SmbConstants static final LinkedList CONNECTIONS = new LinkedList(); Hmm, where is this used at? Doing a quick search for all occurrences of CONNECTIONS on the jcifs code produces only one class, SmbTransport. All of the references I found were in one method, getSmbTransport() [lines 43-64], and it only added to the LinkedList. I could not find any place where the connections are removed from the LinkedList. Eventually the list grows to where it consumes enough memory to cause a java.lang.OutOfMemoryError condition. The error will appear faster if the jcifs.smb.client.ssnLimit=1 property is set. Andy |
|
|
Re: Memory Leak?
by Michael B Allen
::
Rate this Message:
On Thu, Sep 17, 2009 at 10:34 AM, Andy Thomson <a10008051@...> wrote:
> Mike, > > Seeing a memory leak in the latest jcifs 1.3.12 or it sure looks like one. > > This is from a long running process -- typically after about 10 hours it > will error with an out of memory condition. > > System has 64GB memory > java -Xmx1024m > jcifs.smb.client.ssnLimit=1 > > "Exception in thread "Transport414026" java.lang.OutOfMemoryError: Java > heap space" > > I happened to grab a heap dump about 11 hours into the latest run, > ironically it ran out of memory shortly afterwards. What it looks like > is that the connections are never released. More about this after some > of the statistical information from the jhat tool. With ssnLimit=1 I'm not surprised by this at all. Setting ssnLimit = 1 causes JCIFS to create a completely new connection for each SmbFile. I'm surprised it ran as long as it did. However, after the transport has been idle for soTimeout, it should release the socket and all sessions on that transport (which will be only 1 if ssnLimit = 1) and all trees for those sessions etc so it should recover much of those resources. > Listed below is a partial list containing a count of the all instances > in the dump, this is just the top of the list. After 11 hours of running > there are 304,000+ instances of various jcifs objects. Notice that the > one of the largest consumers is a LinkedList. These objects account for > 739 MB of "retained" memory in this heap dump. > > Instance Counts for All Classes (including platform) > > 1519162 instances of class [Ljava.lang.Object; > 1236988 instances of class java.lang.Object > 1219633 instances of class java.util.LinkedList$Entry > 630506 instances of class java.lang.String > 630493 instances of class [C > 609819 instances of class java.util.LinkedList > 607371 instances of class [S > 313855 instances of class [B > 312116 instances of class java.net.Inet4Address > 310583 instances of class java.net.Socket > 310583 instances of class java.net.SocksSocketImpl > 305744 instances of class java.lang.ref.Finalizer > 305253 instances of class [Ljava.util.HashMap$Entry; > 305246 instances of class java.util.HashMap > 304934 instances of class java.util.Vector > 304925 instances of class jcifs.smb.SmbComBlankResponse > 304908 instances of class jcifs.smb.SmbTransport > 304908 instances of class jcifs.smb.SmbTransport$ServerData > 304907 instances of class jcifs.UniAddress > 304907 instances of class jcifs.smb.SmbSession > 304907 instances of class jcifs.smb.SmbTree > 302970 instances of class java.net.ConnectException > 302695 instances of class jcifs.util.transport.TransportException > > An outline of the code is here, it's not really relevant, more for > reference. It just chains an SmbFile opening a server/share, then a > directory, and finally a file, which is written to using > SmbFileOutputStream. Everything is in try {} catch {} finally {} blocks > to make sure nothing escapes :-). This is most important for the > stream to insure its closed no matter what. > > SmbFile targetServer = new SmbFile(machinePath, auth); > > do some stuff > > Smbfile targetDirectory = new SmbFile(smbTargetServer, directory); > > do more stuff > > SmbFile smbTargetFile = new SmbFile(smbTargetDir, fileName); > > do even more stuff > > SmbFileOutputStream o =null; > try { > o = new SmbFileOutputStream(smbTargetFile); > write(); > catch { > } finally { > try { > o.close(); > catch (Exception ex) { > } > } > > --- > > Going back to the heap dump, one of the biggest consumers is a > LinkedList. Drilling into the heap, the entries are from SmbConstants > and the list contains a reference called CONNECTIONS. Looking at the > source file indicates that there is indeed a LinkedList called CONNECTIONS. > > jcifs.smb.SmbConstants > > static final LinkedList CONNECTIONS = new LinkedList(); > > Hmm, where is this used at? Doing a quick search for all occurrences of > CONNECTIONS on the jcifs code produces only one class, SmbTransport. > > All of the references I found were in one method, getSmbTransport() > [lines 43-64], and it only added to the LinkedList. I could not find > any place where the connections are removed from the LinkedList. > Eventually the list grows to where it consumes enough memory to cause a > java.lang.OutOfMemoryError condition. The error will appear faster if > the jcifs.smb.client.ssnLimit=1 property is set. Entries in the CONNECTIONS list should be removed at some point but currently they are not. This is a bug. There should be some expiration logic. However, it is a minor bug because you really should not set ssnLimit = 1. Creating a separate connection for each session is just an enormous waste of resources as you have discovered. With ssnLimit = 1 the CONNECTIONS list does effectively become an object sink (aka memory leak in Java speak). Short Answer: Don't set ssnLimit = 1. Mike -- Michael B Allen Java Active Directory Integration http://www.ioplex.com/ |
| Free embeddable forum powered by Nabble | Forum Help |
