Exceptions during stress test

[yunien]

Hi,

I stress tested Sailfin without having anything deployed. The scenario is as follows:

PC1: INVITE PC2 --> Sailfin
Sailfin: 100 Trying --> PC1
Sailfin: INVITE PC2 --> PC2
PC2: 200 OK --> Sailfin
Sailfin: 200 OK --> PC1

- After some time I get error messages like:

[#|2010-02-22T18:47:24.648+0100|SEVERE|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=31;_ThreadName=SipContainer-serversWorkerThread-5060-5;117505;_RequestID=5152cd90-d28f-42ee-b261-6a9817ea1f2d;|"Cannot find matching client transaction for the response with CallId 117505"|#]

- Afterwards I saw a lot of these:

[#|2010-02-22T18:48:19.367+0100|SEVERE|sun-glassfish-comms-server2.0|grizzly|_ThreadID=28;_ThreadName=SipContainer-serversWorkerThread-5060-7;_RequestID=ef19cf67-da0f-4768-aa5f-46849d5629a2;|WorkerThreadImpl unexpected exception:
java.lang.IllegalStateException: This DialogFragment is in confirmed state and has no longer a DialogSet reference
        at com.ericsson.ssa.sip.DialogFragment.getDialogSet(DialogFragment.java:330)
        at com.ericsson.ssa.sip.DialogSet.registerEarlyDialog(DialogSet.java:280)
        at com.ericsson.ssa.sip.DialogManager.dispatch(DialogManager.java:819)
        at com.ericsson.ssa.sip.LocalRouteManager.dispatch(LocalRouteManager.java:148)
        at com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatchViaStatelessProxy(ApplicationDispatcher.java:638)
        at com.ericsson.ssa.container.sim.ApplicationDispatcher.invokeServletDispatchers(ApplicationDispatcher.java:424)
        at com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:174)
        at com.ericsson.ssa.sip.ProxyBranchImpl$3.call(ProxyBranchImpl.java:724)
        at com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
        at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)
|#]


Somebody knows what could have caused the error messages?

Also, running the stress test for some minutes suddenly caused the CPU to go up to 100% - and never get back to zero, even after waiting 12 hours. How could I find out what thread is using 100% of the CPU? If somebody would like to analyze this from the beginning, I could send out the root login, just let me know!

And one last question: I saw the ConcurrentSipDialogs counter (under "Monitoring") climb high. Is it because I don't send ACKs? Could I somehow lower the timeout until the dialogs get removed?

Thanks a lot!
Florian

[rampsarathy]

Hi Florian,
    You could use JTop to monitor per thread cpu usage, there is a JTop
plugin for JConsole.
Please see the last section  that shows how to start JConsole with JTop
http://java.sun.com/javase/6/docs/technotes/guides/management/jconsole.html

Are you testing a scenario, where the UA does not send an ACK ?

Also, please mention the platform (h/w, os..) and the jvm tunings that
you are using for this test.

Thanks
-Ramesh
yunien wrote:

> Hi,
>
> I stress tested Sailfin without having anything deployed. The scenario is as
> follows:
>
> PC1: INVITE PC2 --> Sailfin
> Sailfin: 100 Trying --> PC1
> Sailfin: INVITE PC2 --> PC2
> PC2: 200 OK --> Sailfin
> Sailfin: 200 OK --> PC1
>
> - After some time I get error messages like:
>
> [#|2010-02-22T18:47:24.648+0100|SEVERE|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=31;_ThreadName=SipContainer-serversWorkerThread-5060-5;117505;_RequestID=5152cd90-d28f-42ee-b261-6a9817ea1f2d;|"Cannot
> find matching client transaction for the response with CallId 117505"|#]
>
> - Afterwards I saw a lot of these:
>
> [#|2010-02-22T18:48:19.367+0100|SEVERE|sun-glassfish-comms-server2.0|grizzly|_ThreadID=28;_ThreadName=SipContainer-serversWorkerThread-5060-7;_RequestID=ef19cf67-da0f-4768-aa5f-46849d5629a2;|WorkerThreadImpl
> unexpected exception:
> java.lang.IllegalStateException: This DialogFragment is in confirmed state
> and has no longer a DialogSet reference
>         at
> com.ericsson.ssa.sip.DialogFragment.getDialogSet(DialogFragment.java:330)
>         at
> com.ericsson.ssa.sip.DialogSet.registerEarlyDialog(DialogSet.java:280)
>         at
> com.ericsson.ssa.sip.DialogManager.dispatch(DialogManager.java:819)
>         at
> com.ericsson.ssa.sip.LocalRouteManager.dispatch(LocalRouteManager.java:148)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatchViaStatelessProxy(ApplicationDispatcher.java:638)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.invokeServletDispatchers(ApplicationDispatcher.java:424)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:174)
>         at
> com.ericsson.ssa.sip.ProxyBranchImpl$3.call(ProxyBranchImpl.java:724)
>         at
> com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
>         at
> com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)
> |#]
>
>
> Somebody knows what could have caused the error messages?
>
> Also, running the stress test for some minutes suddenly caused the CPU to go
> up to 100% - and never get back to zero, even after waiting 12 hours. How
> could I find out what thread is using 100% of the CPU? If somebody would
> like to analyze this from the beginning, I could send out the root login,
> just let me know!
>
> And one last question: I saw the ConcurrentSipDialogs counter (under
> "Monitoring") climb high. Is it because I don't send ACKs? Could I somehow
> lower the timeout until the dialogs get removed?
>
> Thanks a lot!
> Florian
>
>  

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[yunien]

Hi Ramesh,

thanks for info! I'll try with JConsole and check what thread it is.

Exactly, there's no ACK in my scenarion. The application I wrote doesn't have to handle any ACKs or BYEs; after the 200 OK being sent out to PC1, my Sailfin application doesn't have to care about the call any more.

My Sailfin is running on Debian 5.0 and is a Core i7 with 3.33GHz. There are no JVM tunings, I'm running the default standalone instance.

Best Regards,
Florian

rampsarathy wrote:

Hi Florian,
    You could use JTop to monitor per thread cpu usage, there is a JTop
plugin for JConsole.
Please see the last section  that shows how to start JConsole with JTop
http://java.sun.com/javase/6/docs/technotes/guides/management/jconsole.html

Are you testing a scenario, where the UA does not send an ACK ?

Also, please mention the platform (h/w, os..) and the jvm tunings that
you are using for this test.

Thanks
-Ramesh
yunien wrote:
> Hi,
>
> I stress tested Sailfin without having anything deployed. The scenario is as
> follows:
>
> PC1: INVITE PC2 --> Sailfin
> Sailfin: 100 Trying --> PC1
> Sailfin: INVITE PC2 --> PC2
> PC2: 200 OK --> Sailfin
> Sailfin: 200 OK --> PC1
>
> - After some time I get error messages like:
>
> [#|2010-02-22T18:47:24.648+0100|SEVERE|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=31;_ThreadName=SipContainer-serversWorkerThread-5060-5;117505;_RequestID=5152cd90-d28f-42ee-b261-6a9817ea1f2d;|"Cannot
> find matching client transaction for the response with CallId 117505"|#]
>
> - Afterwards I saw a lot of these:
>
> [#|2010-02-22T18:48:19.367+0100|SEVERE|sun-glassfish-comms-server2.0|grizzly|_ThreadID=28;_ThreadName=SipContainer-serversWorkerThread-5060-7;_RequestID=ef19cf67-da0f-4768-aa5f-46849d5629a2;|WorkerThreadImpl
> unexpected exception:
> java.lang.IllegalStateException: This DialogFragment is in confirmed state
> and has no longer a DialogSet reference
>         at
> com.ericsson.ssa.sip.DialogFragment.getDialogSet(DialogFragment.java:330)
>         at
> com.ericsson.ssa.sip.DialogSet.registerEarlyDialog(DialogSet.java:280)
>         at
> com.ericsson.ssa.sip.DialogManager.dispatch(DialogManager.java:819)
>         at
> com.ericsson.ssa.sip.LocalRouteManager.dispatch(LocalRouteManager.java:148)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatchViaStatelessProxy(ApplicationDispatcher.java:638)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.invokeServletDispatchers(ApplicationDispatcher.java:424)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:174)
>         at
> com.ericsson.ssa.sip.ProxyBranchImpl$3.call(ProxyBranchImpl.java:724)
>         at
> com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
>         at
> com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)
> |#]
>
>
> Somebody knows what could have caused the error messages?
>
> Also, running the stress test for some minutes suddenly caused the CPU to go
> up to 100% - and never get back to zero, even after waiting 12 hours. How
> could I find out what thread is using 100% of the CPU? If somebody would
> like to analyze this from the beginning, I could send out the root login,
> just let me know!
>
> And one last question: I saw the ConcurrentSipDialogs counter (under
> "Monitoring") climb high. Is it because I don't send ACKs? Could I somehow
> lower the timeout until the dialogs get removed?
>
> Thanks a lot!
> Florian
>
>  


---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[Jason Hu]

Hi Rampsarathy,

I am encountering same exception as Yunien. When I am putting heavy traffic, it seams that sailfin refuse to process request (usage of CPU and memory is very low, timeout of sip request raises fast). When I reduce traffic to normal level, it can't be recovered.

Could you please help on this? thanks a lot.

Best regard
Jason Hu

rampsarathy wrote:

Hi Florian,
    You could use JTop to monitor per thread cpu usage, there is a JTop
plugin for JConsole.
Please see the last section  that shows how to start JConsole with JTop
http://java.sun.com/javase/6/docs/technotes/guides/management/jconsole.html

Are you testing a scenario, where the UA does not send an ACK ?

Also, please mention the platform (h/w, os..) and the jvm tunings that
you are using for this test.

Thanks
-Ramesh
yunien wrote:
> Hi,
>
> I stress tested Sailfin without having anything deployed. The scenario is as
> follows:
>
> PC1: INVITE PC2 --> Sailfin
> Sailfin: 100 Trying --> PC1
> Sailfin: INVITE PC2 --> PC2
> PC2: 200 OK --> Sailfin
> Sailfin: 200 OK --> PC1
>
> - After some time I get error messages like:
>
> [#|2010-02-22T18:47:24.648+0100|SEVERE|sun-glassfish-comms-server2.0|javax.enterprise.system.container.sip|_ThreadID=31;_ThreadName=SipContainer-serversWorkerThread-5060-5;117505;_RequestID=5152cd90-d28f-42ee-b261-6a9817ea1f2d;|"Cannot
> find matching client transaction for the response with CallId 117505"|#]
>
> - Afterwards I saw a lot of these:
>
> [#|2010-02-22T18:48:19.367+0100|SEVERE|sun-glassfish-comms-server2.0|grizzly|_ThreadID=28;_ThreadName=SipContainer-serversWorkerThread-5060-7;_RequestID=ef19cf67-da0f-4768-aa5f-46849d5629a2;|WorkerThreadImpl
> unexpected exception:
> java.lang.IllegalStateException: This DialogFragment is in confirmed state
> and has no longer a DialogSet reference
>         at
> com.ericsson.ssa.sip.DialogFragment.getDialogSet(DialogFragment.java:330)
>         at
> com.ericsson.ssa.sip.DialogSet.registerEarlyDialog(DialogSet.java:280)
>         at
> com.ericsson.ssa.sip.DialogManager.dispatch(DialogManager.java:819)
>         at
> com.ericsson.ssa.sip.LocalRouteManager.dispatch(LocalRouteManager.java:148)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatchViaStatelessProxy(ApplicationDispatcher.java:638)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.invokeServletDispatchers(ApplicationDispatcher.java:424)
>         at
> com.ericsson.ssa.container.sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:174)
>         at
> com.ericsson.ssa.sip.ProxyBranchImpl$3.call(ProxyBranchImpl.java:724)
>         at
> com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:325)
>         at
> com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:184)
> |#]
>
>
> Somebody knows what could have caused the error messages?
>
> Also, running the stress test for some minutes suddenly caused the CPU to go
> up to 100% - and never get back to zero, even after waiting 12 hours. How
> could I find out what thread is using 100% of the CPU? If somebody would
> like to analyze this from the beginning, I could send out the root login,
> just let me know!
>
> And one last question: I saw the ConcurrentSipDialogs counter (under
> "Monitoring") climb high. Is it because I don't send ACKs? Could I somehow
> lower the timeout until the dialogs get removed?
>
> Thanks a lot!
> Florian
>
>  


---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]