HtmlUnit leaks memory
|
View:
New views
7 Messages
—
Rating Filter:
Alert me
|
 |
HtmlUnit leaks memory
by P H-2
::
Rate this Message:
Hello,
I run HtmlUnit in multiple threads with Grinder loadtesting tool. I monitored the memory usage of the HtmlUnit process and noticed that it grew bigger and bigger during the test. At the beginning the process consumed about 100k of memory, but at the end it went over 1000k. I called the webClient.closeAllWindows() after each thread has finished. Has anyone else noticed the same? pete .................................................................... Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Htmlunit-user mailing list Htmlunit-user@... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
|
Re: HtmlUnit leaks memory
by Marc Guillemot
::
Rate this Message:
Hi,
if you already call closeAllWindows, I fear that I have no other hint for you: you'll have to start profiling to understand where is the leak. It's not impossible that HtmlUnit has some memory leaks but all problems reported in this area during the last months were caused by user errors (like missing closeAllWindows). Cheers, Marc. P H a écrit : > Hello, > > I run HtmlUnit in multiple threads with Grinder loadtesting tool. I monitored the memory usage of the HtmlUnit process and noticed that it grew bigger and bigger during the test. At the beginning the process consumed about 100k of memory, but at the end it went over 1000k. > > I called the webClient.closeAllWindows() after each thread has finished. > > > Has anyone else noticed the same? > > pete > > .................................................................... > Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. > Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > Htmlunit-user mailing list > Htmlunit-user@... > https://lists.sourceforge.net/lists/listinfo/htmlunit-user > ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Htmlunit-user mailing list Htmlunit-user@... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
|
|
|
|
Re: HtmlUnit leaks memory
by Nidhi-5
::
Rate this Message:
I used tptp framework to do some profiling of the code. Its multiple
places. If your client is javascript enabled then initialization (init() method) of JavaScriptEngine.java consumes a lot of memory as it is called for each new window or page I think. Also Interpreterloop method in Interpreter.java (Rhino engine) consumes a lot of memory over all , however its more distributed among lot of small methods that get called during bytecode execution. I tried putting final for int[] , byte[] or string[] declarations whereever possible and that saves some memory but I think mainly javascript engine initialization needs to be optimized when initializing new window. Please share what you guys think. Nidhi P H wrote: > Hi, > > I did not mention that I downloaded the sources of HtmlUnit and made some minor changes to the code, but those should certainly not cause any memory leaks to HtmlUnit. > > Is the downloaded sources the same as the latest version of the HtmlUnit. I had to copy some .properties files to a different location because the WebClient constructor threw an exception because it could locate those files from the path it tried to find them. > > - Pete > > > Marc Guillemot kirjoitti 03.11.2009 kello 10:10: > >> Hi, >> >> if you already call closeAllWindows, I fear that I have no other hint >> for you: you'll have to start profiling to understand where is the leak. >> >> It's not impossible that HtmlUnit has some memory leaks but all >> problems >> reported in this area during the last months were caused by user errors >> (like missing closeAllWindows). >> >> Cheers, >> Marc. >> >> P H a écrit : >> >>> Hello, >>> >>> I run HtmlUnit in multiple threads with Grinder loadtesting tool. I >>> >> monitored the memory usage of the HtmlUnit process and noticed that >> it grew >> bigger and bigger during the test. At the beginning the process consumed >> about 100k of memory, but at the end it went over 1000k. >> >>> I called the webClient.closeAllWindows() after each thread has finished. >>> >>> >>> Has anyone else noticed the same? >>> >>> pete >>> >>> .................................................................... >>> Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. >>> Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku >>> >>> >>> >> ---------------------------------------------------------------------------- >> -- >> >>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >>> is the only developer event you need to attend this year. Jumpstart your >>> developing skills, take BlackBerry mobile applications to market and stay >>> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >>> http://p.sf.net/sfu/devconference >>> _______________________________________________ >>> Htmlunit-user mailing list >>> Htmlunit-user@... >>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >>> >>> >> ---------------------------------------------------------------------------- >> -- >> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >> is the only developer event you need to attend this year. Jumpstart your >> developing skills, take BlackBerry mobile applications to market and >> stay >> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >> http://p.sf.net/sfu/devconference >> _______________________________________________ >> Htmlunit-user mailing list >> Htmlunit-user@... >> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >> > > > .................................................................... > Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. > Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > Htmlunit-user mailing list > Htmlunit-user@... > https://lists.sourceforge.net/lists/listinfo/htmlunit-user > -- ------------------------------------------------ Nidhi Shah Research Scientist Purewire, Inc. 404-963-9025 nidhi@... ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Htmlunit-user mailing list Htmlunit-user@... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
|
Re: HtmlUnit leaks memory
by Marc Guillemot
::
Rate this Message:
Hi,
I don't understand: do you mean memory leak or speed improvement here? I think that it is normal that memory and time is consumed: it has some job to do. The question is only, does it leak or not. Cheers, Marc. Nidhi a écrit : > I used tptp framework to do some profiling of the code. Its multiple > places. If your client is javascript enabled then initialization > (init() method) of JavaScriptEngine.java consumes a lot of memory as it > is called for each new window or page I think. Also Interpreterloop > method in Interpreter.java (Rhino engine) consumes a lot of memory over > all , however its more distributed among lot of small methods that get > called during bytecode execution. I tried putting final for int[] , > byte[] or string[] declarations whereever possible and that saves some > memory but I think mainly javascript engine initialization needs to be > optimized when initializing new window. > > Please share what you guys think. > > Nidhi > > P H wrote: >> Hi, >> >> I did not mention that I downloaded the sources of HtmlUnit and made some minor changes to the code, but those should certainly not cause any memory leaks to HtmlUnit. >> >> Is the downloaded sources the same as the latest version of the HtmlUnit. I had to copy some .properties files to a different location because the WebClient constructor threw an exception because it could locate those files from the path it tried to find them. >> >> - Pete >> >> >> Marc Guillemot kirjoitti 03.11.2009 kello 10:10: >> >>> Hi, >>> >>> if you already call closeAllWindows, I fear that I have no other hint >>> for you: you'll have to start profiling to understand where is the leak. >>> >>> It's not impossible that HtmlUnit has some memory leaks but all >>> problems >>> reported in this area during the last months were caused by user errors >>> (like missing closeAllWindows). >>> >>> Cheers, >>> Marc. >>> >>> P H a écrit : >>> >>>> Hello, >>>> >>>> I run HtmlUnit in multiple threads with Grinder loadtesting tool. I >>>> >>> monitored the memory usage of the HtmlUnit process and noticed that >>> it grew >>> bigger and bigger during the test. At the beginning the process consumed >>> about 100k of memory, but at the end it went over 1000k. >>> >>>> I called the webClient.closeAllWindows() after each thread has finished. >>>> >>>> >>>> Has anyone else noticed the same? >>>> >>>> pete >>>> >>>> .................................................................... >>>> Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. >>>> Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku >>>> >>>> >>>> >>> ---------------------------------------------------------------------------- >>> -- >>> >>>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >>>> is the only developer event you need to attend this year. Jumpstart your >>>> developing skills, take BlackBerry mobile applications to market and stay >>>> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >>>> http://p.sf.net/sfu/devconference >>>> _______________________________________________ >>>> Htmlunit-user mailing list >>>> Htmlunit-user@... >>>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >>>> >>>> >>> ---------------------------------------------------------------------------- >>> -- >>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >>> is the only developer event you need to attend this year. Jumpstart your >>> developing skills, take BlackBerry mobile applications to market and >>> stay >>> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >>> http://p.sf.net/sfu/devconference >>> _______________________________________________ >>> Htmlunit-user mailing list >>> Htmlunit-user@... >>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >>> >> >> .................................................................... >> Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. >> Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku >> >> ------------------------------------------------------------------------------ >> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >> is the only developer event you need to attend this year. Jumpstart your >> developing skills, take BlackBerry mobile applications to market and stay >> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >> http://p.sf.net/sfu/devconference >> _______________________________________________ >> Htmlunit-user mailing list >> Htmlunit-user@... >> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >> > > ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Htmlunit-user mailing list Htmlunit-user@... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
|
Re: HtmlUnit leaks memory
by Nidhi-5
::
Rate this Message:
We achieve both but I would call it a memory leak improvement because it
is still doing the same job in less amount of memory that it consumed before. Also all of this memory should be freed as we execute the line 'WebClient.closeAllWindows()' ... but it does not happen , I would call that memory leak. Ofcourse HtmlUnit has a big job to perform but there are few things like string collection overhead, manipulations on strings (like substring, each time you create a substring in java is probably the same amount of memory consumed as the source string). I am no java expert but I am learning as I go and I think they suggest CharSequence instead. Copy array on temporary array for in the small repeatedly called methods, this all occupies the memory that can not be freed easily and hence result in requirement of the more memory as the process runs longer. Memory leaks ultimately cause slow downs because the garbage collection runs longer to find only small chunks to free. Thanks, Nidhi Marc Guillemot wrote: > Hi, > > I don't understand: do you mean memory leak or speed improvement here? > > I think that it is normal that memory and time is consumed: it has some > job to do. The question is only, does it leak or not. > > Cheers, > Marc. > > Nidhi a écrit : > >> I used tptp framework to do some profiling of the code. Its multiple >> places. If your client is javascript enabled then initialization >> (init() method) of JavaScriptEngine.java consumes a lot of memory as it >> is called for each new window or page I think. Also Interpreterloop >> method in Interpreter.java (Rhino engine) consumes a lot of memory over >> all , however its more distributed among lot of small methods that get >> called during bytecode execution. I tried putting final for int[] , >> byte[] or string[] declarations whereever possible and that saves some >> memory but I think mainly javascript engine initialization needs to be >> optimized when initializing new window. >> >> Please share what you guys think. >> >> Nidhi >> >> P H wrote: >> >>> Hi, >>> >>> I did not mention that I downloaded the sources of HtmlUnit and made some minor changes to the code, but those should certainly not cause any memory leaks to HtmlUnit. >>> >>> Is the downloaded sources the same as the latest version of the HtmlUnit. I had to copy some .properties files to a different location because the WebClient constructor threw an exception because it could locate those files from the path it tried to find them. >>> >>> - Pete >>> >>> >>> Marc Guillemot kirjoitti 03.11.2009 kello 10:10: >>> >>> >>>> Hi, >>>> >>>> if you already call closeAllWindows, I fear that I have no other hint >>>> for you: you'll have to start profiling to understand where is the leak. >>>> >>>> It's not impossible that HtmlUnit has some memory leaks but all >>>> problems >>>> reported in this area during the last months were caused by user errors >>>> (like missing closeAllWindows). >>>> >>>> Cheers, >>>> Marc. >>>> >>>> P H a écrit : >>>> >>>> >>>>> Hello, >>>>> >>>>> I run HtmlUnit in multiple threads with Grinder loadtesting tool. I >>>>> >>>>> >>>> monitored the memory usage of the HtmlUnit process and noticed that >>>> it grew >>>> bigger and bigger during the test. At the beginning the process consumed >>>> about 100k of memory, but at the end it went over 1000k. >>>> >>>> >>>>> I called the webClient.closeAllWindows() after each thread has finished. >>>>> >>>>> >>>>> Has anyone else noticed the same? >>>>> >>>>> pete >>>>> >>>>> .................................................................... >>>>> Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. >>>>> Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku >>>>> >>>>> >>>>> >>>>> >>>> ---------------------------------------------------------------------------- >>>> -- >>>> >>>> >>>>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >>>>> is the only developer event you need to attend this year. Jumpstart your >>>>> developing skills, take BlackBerry mobile applications to market and stay >>>>> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >>>>> http://p.sf.net/sfu/devconference >>>>> _______________________________________________ >>>>> Htmlunit-user mailing list >>>>> Htmlunit-user@... >>>>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >>>>> >>>>> >>>>> >>>> ---------------------------------------------------------------------------- >>>> -- >>>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >>>> is the only developer event you need to attend this year. Jumpstart your >>>> developing skills, take BlackBerry mobile applications to market and >>>> stay >>>> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >>>> http://p.sf.net/sfu/devconference >>>> _______________________________________________ >>>> Htmlunit-user mailing list >>>> Htmlunit-user@... >>>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >>>> >>>> >>> .................................................................... >>> Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. >>> Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku >>> >>> ------------------------------------------------------------------------------ >>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA >>> is the only developer event you need to attend this year. Jumpstart your >>> developing skills, take BlackBerry mobile applications to market and stay >>> ahead of the curve. Join us from November 9 - 12, 2009. Register now! >>> http://p.sf.net/sfu/devconference >>> _______________________________________________ >>> Htmlunit-user mailing list >>> Htmlunit-user@... >>> https://lists.sourceforge.net/lists/listinfo/htmlunit-user >>> >>> >> > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Htmlunit-user mailing list > Htmlunit-user@... > https://lists.sourceforge.net/lists/listinfo/htmlunit-user > -- ------------------------------------------------ Nidhi Shah Research Scientist Purewire, Inc. 404-963-9025 nidhi@... ---------------------------------- Check out the Barracuda Spam & Virus Firewall - offering the fastest virus & malware protection in the industry: www.barracudanetworks.com/spam ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Htmlunit-user mailing list Htmlunit-user@... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
|
Re: HtmlUnit leaks memory
by Marc Guillemot
::
Rate this Message:
Hi Nidhi,
patch with improvements are always welcome. Cheers, Marc. Nidhi a écrit : > We achieve both but I would call it a memory leak improvement because it > is still doing the same job in less amount of memory that it consumed > before. Also all of this memory should be freed as we execute the line > 'WebClient.closeAllWindows()' ... but it does not happen , I would call > that memory leak. Ofcourse HtmlUnit has a big job to perform but there > are few things like string collection overhead, manipulations on strings > (like substring, each time you create a substring in java is probably > the same amount of memory consumed as the source string). I am no java > expert but I am learning as I go and I think they suggest CharSequence > instead. Copy array on temporary array for in the small repeatedly > called methods, this all occupies the memory that can not be freed > easily and hence result in requirement of the more memory as the process > runs longer. Memory leaks ultimately cause slow downs because the > garbage collection runs longer to find only small chunks to free. > Thanks, > Nidhi > > > > Marc Guillemot wrote: >> Hi, >> >> I don't understand: do you mean memory leak or speed improvement here? >> >> I think that it is normal that memory and time is consumed: it has some >> job to do. The question is only, does it leak or not. >> >> Cheers, >> Marc. >> >> Nidhi a écrit : >> >>> I used tptp framework to do some profiling of the code. Its multiple >>> places. If your client is javascript enabled then initialization >>> (init() method) of JavaScriptEngine.java consumes a lot of memory as it >>> is called for each new window or page I think. Also Interpreterloop >>> method in Interpreter.java (Rhino engine) consumes a lot of memory over >>> all , however its more distributed among lot of small methods that get >>> called during bytecode execution. I tried putting final for int[] , >>> byte[] or string[] declarations whereever possible and that saves some >>> memory but I think mainly javascript engine initialization needs to be >>> optimized when initializing new window. >>> >>> Please share what you guys think. >>> >>> Nidhi >>> >>> P H wrote: >>> >>>> Hi, >>>> >>>> I did not mention that I downloaded the sources of HtmlUnit and made some minor changes to the code, but those should certainly not cause any memory leaks to HtmlUnit. >>>> >>>> Is the downloaded sources the same as the latest version of the HtmlUnit. I had to copy some .properties files to a different location because the WebClient constructor threw an exception because it could locate those files from the path it tried to find them. >>>> >>>> - Pete >>>> >>>> >>>> Marc Guillemot kirjoitti 03.11.2009 kello 10:10: >>>> >>>> >>>>> Hi, >>>>> >>>>> if you already call closeAllWindows, I fear that I have no other hint >>>>> for you: you'll have to start profiling to understand where is the leak. >>>>> >>>>> It's not impossible that HtmlUnit has some memory leaks but all >>>>> problems >>>>> reported in this area during the last months were caused by user errors >>>>> (like missing closeAllWindows). >>>>> >>>>> Cheers, >>>>> Marc. >>>>> >>>>> P H a écrit : >>>>> >>>>> >>>>>> Hello, >>>>>> >>>>>> I run HtmlUnit in multiple threads with Grinder loadtesting tool. I >>>>>> >>>>>> >>>>> monitored the memory usage of the HtmlUnit process and noticed that >>>>> it grew >>>>> bigger and bigger during the test. At the beginning the process consumed >>>>> about 100k of memory, but at the end it went over 1000k. >>>>> >>>>> >>>>>> I called the webClient.closeAllWindows() after each thread has finished. >>>>>> >>>>>> >>>>>> Has anyone else noticed the same? >>>>>> >>>>>> pete >>>>>> ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Htmlunit-user mailing list Htmlunit-user@... https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
| Free embeddable forum powered by Nabble | Forum Help |
