[jira] Created: (HTTPCORE-155) requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds
|
View:
New views
20 Messages
—
Rating Filter:
Alert me
|
| < Prev | 1 - 2 - 3 - 4 | Next > |
 |
[jira] Created: (HTTPCORE-155) requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds
by JIRA jira@apache.org
::
Rate this Message:
requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds
---------------------------------------------------------------------------------------------------------------- Key: HTTPCORE-155 URL: https://issues.apache.org/jira/browse/HTTPCORE-155 Project: HttpComponents HttpCore Issue Type: Bug Components: HttpCore NIO Affects Versions: 4.0-beta1 Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking Reporter: Tom McSorley I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? Here's the JVM related thread information: The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) .... (non important stack information removed) 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) Here's the monitor that this thread is blocked and waiting on: 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 3LKWAITERQ Waiting to enter: 3LKWAITER "pool-2-thread-5" (0x2AEECE00) And here's the thread that currently has this monitor locked: 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom McSorley updated HTTPCORE-155: ---------------------------------- Description: I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? Here's the JVM related thread information: The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) .... (non important stack information removed) 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) Here's the monitor that this thread is blocked and waiting on: 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 3LKWAITERQ Waiting to enter: 3LKWAITER "pool-2-thread-5" (0x2AEECE00) And here's the thread that currently has this monitor locked: 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... was: I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? Here's the JVM related thread information: The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) .... (non important stack information removed) 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) Here's the monitor that this thread is blocked and waiting on: 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 3LKWAITERQ Waiting to enter: 3LKWAITER "pool-2-thread-5" (0x2AEECE00) And here's the thread that currently has this monitor locked: 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds > ---------------------------------------------------------------------------------------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Commented: (HTTPCORE-155) requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579644#action_12579644 ] Oleg Kalnichevski commented on HTTPCORE-155: -------------------------------------------- Tom, You see, the threads get locked somewhere deep inside Sun's private classes. This does not seem like a threading issue in HttpCore itself. I suspect this may be a JRE issue and would recommend upgrading to the latest update (1.6.0.5) of JSE 6.0 Oleg > requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds > ---------------------------------------------------------------------------------------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Commented: (HTTPCORE-155) requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579951#action_12579951 ] Tom McSorley commented on HTTPCORE-155: --------------------------------------- Oleg, Actually I was using an IBM JVM initially... I checked and it appears that I'm running the latest-and-greatest version available from IBM. So I switched JVM's... and have now installed a Sun 1.6.0_05 JVM... and everything is now working flawlessly! Thank You! - tom > requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds > ---------------------------------------------------------------------------------------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski updated HTTPCORE-155: --------------------------------------- Fix Version/s: 4.0-rc1 Summary: Performance issues with IBM JRE 6.0 (was: requestOutput() call stalls for long periods of time which delays an HTTP Request for anywhere from 5-60 seconds) I am glad the problem has been resolved. I am concerned, though, there appear to be performance issues with IBM's JRE implementations. Would it be a big deal for you to invest some time into trying out the latest build of IBM JRE 1.5? Oleg > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.0-rc1 > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Commented: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580067#action_12580067 ] Tom McSorley commented on HTTPCORE-155: --------------------------------------- Oleg, I will run each version of the IBM JVM's I have access to and report on the results to you... I will try to get this done within the next day. Regards, - tom > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.0-rc1 > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asankha C. Perera updated HTTPCORE-155: --------------------------------------- Attachment: javacore.20081203.153723.32300.0001.txt I am seeing a deadlock with the IBM JDK with Apache Synapse, which leads to very low performance. This is not seen with the Sun JDK of the same version. 1LKDEADLOCK Deadlock detected !!! NULL --------------------- NULL 2LKDEADLOCKTHR Thread "http-Listener I/O dispatcher-1" (0x0000000000765600) 3LKDEADLOCKWTR is waiting for: 4LKDEADLOCKMON sys_mon_t:0x00007F5B9CF59490 infl_mon_t: 0x00007F5B9CF59510: 4LKDEADLOCKOBJ sun/nio/ch/SelectionKeyImpl@00007F5BA4D11160/00007F5BA4D11178: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "HttpClientWorker-4" (0x00007F5B9CF21100) 3LKDEADLOCKWTR which is waiting for: 4LKDEADLOCKMON sys_mon_t:0x0000000000D0D668 infl_mon_t: 0x0000000000D0D6E8: 4LKDEADLOCKOBJ java/util/Collections$UnmodifiableSet@00007F5BA41A1BB0/00007F5BA41A1BC8: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "http-Listener I/O dispatcher-1" (0x0000000000765600) NULL > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.0-rc1 > > Attachments: javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski updated HTTPCORE-155: --------------------------------------- Fix Version/s: (was: 4.0-rc1) 4.1 I suspect the cause of the problem is the same SelectionKey thread safety issue discussed here [1] Any attempt at working the problem around would be a major change with a negative effect on the stability on the code base. Besides, I am still of an opinion this is a bug in the JRE implementation. I think we should release HttpCore 4.0 as is, and revisit this problem in the course of 4.1 development. Oleg [1] http://www.nabble.com/async-http-clients-td11569193.html#a15029766 > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Commented: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653688#action_12653688 ] Marc Beyerle commented on HTTPCORE-155: --------------------------------------- Hi, folks! My name is Marc Beyerle and I am from IBM. I'm not from the JVM development, so please don't blame me :-) I am from an IBM System z (aka mainframe) customer center, and one of our customers is currently running into the above described problem. They also use Apache Synapse with the IBM JVM, like Asankha. I dug into this and found that the root cause is a "specification hole". If you read the Sun API here: http://java.sun.com/j2se/1.5.0/docs/api/java/nio/channels/SelectionKey.html you will find this: "... In a naive implementation, reading or writing the interest set may block indefinitely if a selection operation is already in progress". And exactly this is happening here. While org.apache.http.impl.nio.reactor.AbstractIOReactor does a select(), the IOSession cannot perform an interestOps(ops) operation. My suggestion is to synchronize around this, such that a deadlock does not occur. I wrote a fix for this against 4.0-beta1 and I will attach the files to this issue. I tested it with both the IBM JVM and the Sun JVM and it works just fine. You will notice that I had to reduce the timeout for the select() operation down to the minimum (1ms), in order to make things reactive. Also, before each select() operation, I check whether one of the IOSessions is waiting to perform an interestOps(ops) operation. In the loop around select(), I start looking for waiting IOSessions after the last waiting one, in order not to privilege the IOSessions at the beginning of the Iterator. I really hope you find this patch useful and you cannot imagine how much work it took me to get all the IBM approvals to post the patch to this issue here. Cheers, Marc > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Attachment: IOSessionImpl.java This patch is for 4.0-beta1, but I think you will get the idea. > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Attachment: IOSessionImpl.java This patch is for 4.0-beta1, but I think you will get the idea. > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Attachment: AbstractIOReactor.java This patch is for 4.0-beta1, but I think you will get the idea. > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Attachment: (was: IOSessionImpl.java) > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Comment: was deleted > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Commented: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653763#action_12653763 ] Asankha C. Perera commented on HTTPCORE-155: -------------------------------------------- Hi Marc Its great to be able to get in touch with someone from IBM, since this is quite important to at least a few IBM customers. I assumed this to be fixable as well, but didn't have the time yet to dig deeper into this. I appreciate your patch and the work behind, and especially getting the approval to share this back with us. I will test this and get back to you. many thanks asankha > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Commented: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653773#action_12653773 ] Oleg Kalnichevski commented on HTTPCORE-155: -------------------------------------------- Marc, I am very appreciative of the time and effort you have put into this patch. I understand it must be difficult for an IBM staffer to get permissions to contribute code developed internally to an open-source project. While I think the patch represents a feasible workaround, I have a number of concerns regarding the impact of the proposed changes on performance of the NIO transport. Firstly, reducing select interval to 1ms effectively makes I/O dispatch threads run in a tight loop this putting unnecessary high load on CPU(s). Secondly, I am concerned about over-synchronization in the IOSessionImpl. A better approach to solving this problem would be a queuing mechanism for modifications of interest ops similar to that for new sessions. The main idea is to ensure that any changes to the interest ops take place on the I/O dispatch thread after the I/O select operation. That would eliminate the need for synchronization altogether. Having said all that, I DO think the fix ought to be made where it is due, that is IBM JRE code. PS: Could you please submit the patch in udiff format? > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Attachment: AbstractIOReactor.diff > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.diff, AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Commented: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653825#action_12653825 ] Marc Beyerle commented on HTTPCORE-155: --------------------------------------- Hi Oleg, I will attach the patch to this JIRA Issue in udiff format. Sorry, my editor was configured to remove EOL whitespace, so the patch is a bit bigger than expected. And yes, my concern was also performance. But when I monitor my system, I do not see a lot of CPU activity, even with this tight loop... If the underlying native implementation of Java NIO is based on polling, then this is simply expected behavior. I did experiment with polling under C a while ago, and this caused virtually no CPU at all. Your idea with using a queueing mechanism sounds very attractive to me :-) However, I have already spent quite a bit of time for all this, so I don't know if I can spent even more... And when I talked to the JVM support, they explained to me that the behavior, which is described above, is indeed needed just in the way it is. Thanks for your feedback & Cheers, Marc > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.diff, AbstractIOReactor.java, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Attachment: IOSessionImpl.java > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.diff, AbstractIOReactor.java, IOSessionImpl.diff, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
|
|
[jira] Updated: (HTTPCORE-155) Performance issues with IBM JRE 6.0
by JIRA jira@apache.org
::
Rate this Message:
[ https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Beyerle updated HTTPCORE-155: ---------------------------------- Attachment: (was: IOSessionImpl.java) > Performance issues with IBM JRE 6.0 > ----------------------------------- > > Key: HTTPCORE-155 > URL: https://issues.apache.org/jira/browse/HTTPCORE-155 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO > Affects Versions: 4.0-beta1 > Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking > Reporter: Tom McSorley > Fix For: 4.1 > > Attachments: AbstractIOReactor.diff, AbstractIOReactor.java, IOSessionImpl.diff, IOSessionImpl.java, javacore.20081203.153723.32300.0001.txt > > > I'm issuing a second HTTP Request on a connection that has very recently returned a null for the submitRequest() call... this 2nd request is being issued approximately 500ms after the submitRequest() null is returned... so the connection has just been established, an HTTP Request/Response-200 cycle has completed just prior to this 2nd request being issued. I'm seeing unusually long delays in the requestOutput() call (verified by surrounding timing prints)... that can range anywhere from a few milliseconds on up to 60 seconds... It eventually unwinds, and then the submitRequest() is called... this 2nd request is dispatched and works fine... but, it is delayed considerably... Is this a known issue and is there a possible work-around? > Here's the JVM related thread information: > The thread being delayed and stuck in the requestOutput() call for a long time (mostly longer than 5 seconds): > 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8, state:B, prio=5 > 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113) > 4XESTACKTRACE at org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158) > .... (non important stack information removed) > 4XESTACKTRACE at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > Here's the monitor that this thread is blocked and waiting on: > 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30: > 3LKMONOBJECT sun/nio/ch/Util$1@00B09208/00B09214: Flat locked by "I/O dispatcher 7" (0x2A208E00), entry count 1 > 3LKWAITERQ Waiting to enter: > 3LKWAITER "pool-2-thread-5" (0x2AEECE00) > And here's the thread that currently has this monitor locked: > 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C, state:R, prio=5 > 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5, native policy:UNKNOWN) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code)) > 4XESTACKTRACE at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code)) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70) > 4XESTACKTRACE at org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318) > 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735) > I should also note that we're attempting to use 1000 client instances on this single system... each with potentially 2 active connections simultaneously... there is also virtually no CPU load (i.e. less then 5%) on this system... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@... For additional commands, e-mail: dev-help@... |
| < Prev | 1 - 2 - 3 - 4 | Next > |
| Free embeddable forum powered by Nabble | Forum Help |
