[JIRA] Created: (CC-887) OutOfMemory during log_merge

View: New views
11 Messages — Rating Filter:   Alert me  

[JIRA] Created: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

OutOfMemory during log_merge
----------------------------

                 Key: CC-887
                 URL: http://jira.public.thoughtworks.org/browse/CC-887
             Project: CruiseControl
          Issue Type: Bug
          Components: Core Application
    Affects Versions: 2.8.2, 2.8.1, 2.7.3, 2.7.2
         Environment: Windows XP Pro SP2
            Reporter: Matthew Choppen
            Priority: Blocker


during log merge the cruise control java process memory increases (larger the logfile the larger the leak)

<log dir="${cruisecontrol.destination}\logs\${project.name}\">
  <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
</log>

We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....

This is the last entry in the cruisecontrol.log file:
2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points

We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold)

Here is the Console output:

WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop







--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Updated: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


     [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew Choppen updated CC-887:
-------------------------------

    Description:
during log merge the cruise control java process memory increases (larger the logfile the larger the leak)

<log dir="${cruisecontrol.destination}\logs\${project.name}\">
  <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
</log>

We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....

This is the last entry in the cruisecontrol.log file:
2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points

We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM

Here is the Console output:

WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop







  was:
during log merge the cruise control java process memory increases (larger the logfile the larger the leak)

<log dir="${cruisecontrol.destination}\logs\${project.name}\">
  <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
</log>

We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....

This is the last entry in the cruisecontrol.log file:
2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points

We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold)

Here is the Console output:

WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:619)
09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop








> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17254 ]

Archimedes Trajano commented on CC-887:
---------------------------------------

Can the implementation be changed so that the merge is done through a separately spawned JVM?  That would prevent the log merge from killing the rest of the application.

> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17289 ]

Michael Kloss commented on CC-887:
----------------------------------

Hmm, as far as I know the log xml is still build up using jdom. This will keep all log data in Memory. Maybe it is a good idea to switch to stax and write the log out straight forward. I will take a look at the code tomorrow to see how much effort this will be to implement and if this could be done rather easily.
( I have made really good experience using stax instead of jdom on my last projects)

> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17810 ]

Brandon Petty commented on CC-887:
----------------------------------

I am having the same issue using distributed CC 2.8.2 under HP-UX 11i.
I am not seeing this with Linux, Solaris, or AIX.

It ends up crashing my remote build agent.  Actually, it is still running... some jini thread must be dead because it disconnects and I have to shut the entire thing down.

     [java]     ... 5 more
     [java] Caused by: java.io.IOException: could not create I/O threads
     [java]     at com.sun.jini.jeri.internal.mux.StreamConnectionIO.start(StreamConnectionIO.java:107)
     [java]     at com.sun.jini.jeri.internal.mux.Mux.start(Mux.java:196)
     [java]     at net.jini.jeri.connection.ConnectionManager$OutboundMux.newRequest(ConnectionManager.java:356)
     [java]     at net.jini.jeri.connection.ConnectionManager$ReqIterator.next(ConnectionManager.java:630)
     [java]     at net.jini.jeri.BasicObjectEndpoint$1.next(BasicObjectEndpoint.java:371)
     [java]     at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:708)
     [java]     ... 9 more
     [java] Caused by: java.lang.OutOfMemoryError: unable to create new native thread
     [java]     at java.lang.Thread.start0(Native Method)
     [java]     at java.lang.Thread.start(Thread.java:597)
     [java]     at com.sun.jini.thread.ThreadPool.execute(ThreadPool.java:101)
     [java]     at com.sun.jini.jeri.internal.mux.StreamConnectionIO.start(StreamConnectionIO.java:98)
     [java]     ... 14 more
     [java] [cc]Sep-25 08:09:24 ZipUtil       - Zipping files from: /disk2/usr/local/cruisecontrol-bin-2.8.2/CCAgent/logs/HPUX-NightlyBuild to: /disk2/usr/local/cruisecontrol-bin-2.8.2/CCAgent/HPUX-NightlyBuild-logs-7801459058557136325.zip
     [java] [cc]Sep-25 08:09:24 ZipUtil       - Finished zipping files
     [java] [cc]Sep-25 08:09:24 ZipUtil       - Zipping files from: /disk2/usr/local/cruisecontrol-bin-2.8.2/CCAgent/output/HPUX-NightlyBuild to: /disk2/usr/local/cruisecontrol-bin-2.8.2/CCAgent/HPUX-NightlyBuild-output-2110212628012449100.zip
     [java] [cc]Sep-25 08:09:24 ZipUtil       - Finished zipping files
     [java] [cc]Sep-25 08:09:28 entServiceImpl- Agent Host: surveyor; agent busy status changed to: false

> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17854 ]

David Gonzales commented on CC-887:
-----------------------------------

I am getting the same problem, I have cruise control set up using ANT 1.7.0 we generate a lot of log data that we merge into the file, I have set ANT_OPTS and looked at the testdetails.xsl. This only delays the problem of the memory leak, is there a solution for this yet?

> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17860 ]

Dan Rollo commented on CC-887:
------------------------------

As Siddharth mentioned, this may be an inherent design issue for really large log files.
Patches to address this are certainly welcome. When writing a patch, be sure to use the latest svn source, as it fixed some other Dashboard related memory issues.

> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17865 ]

Jeffrey Fredrick commented on CC-887:
-------------------------------------

I'm not convinced this is related to large test files. There was an issue prior to 2.6.2 when we used Xalan but as far as I know that went away when we merged the fix described on this page (replace Xalan with Saxon; changes to test details):

http://confluence.public.thoughtworks.org/display/CC/OutOfMemoryError

This would let us merge files that were 10 MB and more.  The people who are seeing crashes, how large are your log files?

Historically there could be a problem with even with a large value of -Xmx (maximum heap size) if the value of -Xms (initial heap size) was too low. Raising -Xms would fix that problem.

Looking for this error message on the mailing list I found this thread:

http://www.nabble.com/CruiseControl-Dashboard-issue.-td23335658.html

There the poster said they were able to solve their OOM Errors by using:

set CC_OPTS=-server -Xms1024m -Xmx1024m -XX:+AggressiveOpts

The reporter of the linked bug (CC-880) also said he was able to solve his problem by increasing max memory.

(David Gonzales: ANT_OPTS won't help with crashes merging the logs. You'll need to change the settings for CC not Ant.)

That's not to say that there isn't a bug here, but we need more information from the people who are experiencing it. Details include:

     version of CruiseControl
     Stacktraces
     value of CC_OPTS
     size of log files
     size of files being merged into the log files
     JVM version & platform

Hopefully this will let us triangulate onto the cause of the problem.

Oh, and people might also want to try the 2.8.3-dev build that is now available on SF if only for the fix to CC-831 : https://sourceforge.net/projects/cruisecontrol/files/CruiseControl/2.8.3/



> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17867 ]

David Gonzales commented on CC-887:
-----------------------------------

Cruse Control version 2.8.2

java.lang.OutOfMemoryError: Java heap space
        at org.apache.jasper.runtime.BodyContentImpl.reAllocBuff(BodyContentImpl.java:626)
        at org.apache.jasper.runtime.BodyContentImpl.write(BodyContentImpl.java:133)
        at net.sourceforge.cruisecontrol.taglib.XSLTag.copy(XSLTag.java:195)
        at net.sourceforge.cruisecontrol.taglib.XSLTag.serveCachedCopy(XSLTag.java:176)
        at net.sourceforge.cruisecontrol.taglib.XSLTag.doEndTag(XSLTag.java:304)
        at org.apache.jsp.main_jsp._jspx_meth_cruisecontrol_xsl_1(org.apache.jsp.main_jsp:1079)
        at org.apache.jsp.main_jsp._jspService(org.apache.jsp.main_jsp:460)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:93)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:373)
        at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:470)
        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:364)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:829)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)

over 1,000 log.xml file totalling 1GB, file size varies depending on the build size, largest is 1,600 KB smallest 450KB

Java(TM) SE Runtime Environment (build 1.6.0_12-b04) win XP SP2



> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17869 ]

Jeffrey Fredrick commented on CC-887:
-------------------------------------

David, thanks for your quick response.

1.6 MB is not a particularly large log file so I think we can safely say that you aren't hitting a limit based on the file size.

Are you using the binary release or the Windows installer?

If the binary release then take a look at cruisecontrol.bat and you should find these lines:

REM Uncomment the following line if you have OutOfMemoryError errors
REM set CC_OPTS=-Xms128m -Xmx256m

Try removing the REM on the set CC_OPTS line.

If you're using the Windows installer then you it is using a service wrapper that is configured by the file wrapper.conf.  Find that file and look for the lines:

 # Initial Java Heap Size (in MB)
 wrapper.java.initmemory=3

 # Maximum Java Heap Size (in MB)
 wrapper.java.maxmemory=256

Try increasing those values (the initmemory should probably be at least 1/2 of max memory) and restarting the CC service.

I strongly suspect that giving more memory to CC will solve your problem.

> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel

[JIRA] Commented: (CC-887) OutOfMemory during log_merge

by JIRA jira@users.public.thoughtworks.org :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


    [ http://jira.public.thoughtworks.org/browse/CC-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_17870 ]

David Gonzales commented on CC-887:
-----------------------------------

Using the binary release so I will try setting the CC_OPTS line and run it.

Thanks.

> OutOfMemory during log_merge
> ----------------------------
>
>                 Key: CC-887
>                 URL: http://jira.public.thoughtworks.org/browse/CC-887
>             Project: CruiseControl
>          Issue Type: Bug
>          Components: Core Application
>    Affects Versions: 2.7.2, 2.7.3, 2.8.1, 2.8.2
>         Environment: Windows XP Pro SP2
>            Reporter: Matthew Choppen
>            Priority: Blocker
>
> during log merge the cruise control java process memory increases (larger the logfile the larger the leak)
> <log dir="${cruisecontrol.destination}\logs\${project.name}\">
>   <merge file="D:\Data\CCase\Build\NightlyBuild\TESTS-TestSuites.xml" />
> </log>
> We found some info about testdetails.xsl and swapped it out which seemed to improve the memory leak but did not fix it.....
> This is the last entry in the cruisecontrol.log file:
> 2009-02-09 11:32:22,538 [btpool0-4] WARN  GetProjectBuildStatusController - java.lang.RuntimeException: java.lang.NumberFormatException: multiple points
> We have forked the processes, increased memory to the VM (it crashes before we reach the threshhold), I also have available hs_err logs from JVM
> Here is the Console output:
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop
> WARNUNG: RMI TCP Accept-0: accept loop for ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=1205] throws
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:597)
>         at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727)
>         at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:384)
>         at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
>         at java.lang.Thread.run(Thread.java:619)
> 09.02.2009 11:44:53 sun.rmi.transport.tcp.TCPTransport$AcceptLoop executeAcceptLoop

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.public.thoughtworks.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cruisecontrol-devel mailing list
Cruisecontrol-devel@...
https://lists.sourceforge.net/lists/listinfo/cruisecontrol-devel