POST request hangs hup

[Ivan Longhi]

Hi,
I'm running tomcat 6.0.18 with java 1.6.0_13 on a linux box.
The only webapp deployed is Sun OpenSSO (http://www.opensso.org) built from
source (20090930).
Sometimes a tomcat thread hangs hup while serving a POST request
(/UI/Login): monitoring tomcat
status I see the processing time grows and the thread never ends. When this
happens I am not able to shutdown
tomcat with shutdown.sh and I need to kill -9 the java process.
I think it is not a problem of memory exausted because if I run "top" the
memory usage by java process
is ok (a few hundreds MB despite CATALINA_OPTS="-Xmx6048M") but the cpu
usage is 100%.
Is there a way to kill a single thread in tomcat whitout restarting it?
Is there a way to limit the max execution time for a single thread/request?

thanks,
ivan

--
ciao,
ivan

[Daniel Henrique Alves Lima]

On Thu, 2009-07-02 at 23:40 +0200, Ivan Longhi wrote:
> Hi,

        Hi, Ivan.

> I'm running tomcat 6.0.18 with java 1.6.0_13 on a linux box.
> The only webapp deployed is Sun OpenSSO (http://www.opensso.org) built from
> source (20090930).
> Sometimes a tomcat thread hangs hup while serving a POST request
> (/UI/Login): monitoring tomcat
> status I see the processing time grows and the thread never ends. When this
> happens I am not able to shutdown
> tomcat with shutdown.sh and I need to kill -9 the java process.
> I think it is not a problem of memory exausted because if I run "top" the
> memory usage by java process
> is ok (a few hundreds MB despite CATALINA_OPTS="-Xmx6048M") but the cpu
> usage is 100%.

        I think could be anything: An infinite loop in your app, a database (or
other resource) deadlock, any intensive db query (with a really long
transaction timeout).
        For instance, last month we've adopted a new Transaction Manager that
keeps some threads (non daemons) running and prevents Tomcat of doing a
"clean" stop. Now i'm "forcing" Tomcat to stop after 5 minutes waiting.

        My (little) experience tells me that in cases like this, your app is
more guilt than your Application Server/Web Container. But it is just my
2 cents.


        []'s


>
--
"If there must be trouble, let it be in my day,
 that my child may have peace."

Thomas Paine


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Daniel Henrique Alves Lima]

is "probably" more guilt :-)


On Thu, 2009-07-02 at 19:20 -0300, Daniel Henrique Alves Lima wrote:

> On Thu, 2009-07-02 at 23:40 +0200, Ivan Longhi wrote:
> > Hi,
>
> Hi, Ivan.
>
> > I'm running tomcat 6.0.18 with java 1.6.0_13 on a linux box.
> > The only webapp deployed is Sun OpenSSO (http://www.opensso.org) built from
> > source (20090930).
> > Sometimes a tomcat thread hangs hup while serving a POST request
> > (/UI/Login): monitoring tomcat
> > status I see the processing time grows and the thread never ends. When this
> > happens I am not able to shutdown
> > tomcat with shutdown.sh and I need to kill -9 the java process.
> > I think it is not a problem of memory exausted because if I run "top" the
> > memory usage by java process
> > is ok (a few hundreds MB despite CATALINA_OPTS="-Xmx6048M") but the cpu
> > usage is 100%.
>
> I think could be anything: An infinite loop in your app, a database (or
> other resource) deadlock, any intensive db query (with a really long
> transaction timeout).
> For instance, last month we've adopted a new Transaction Manager that
> keeps some threads (non daemons) running and prevents Tomcat of doing a
> "clean" stop. Now i'm "forcing" Tomcat to stop after 5 minutes waiting.
>
> My (little) experience tells me that in cases like this, your app is
> more guilt than your Application Server/Web Container. But it is just my
> 2 cents.
>
>
> []'s
>
>
> >

--
"If there must be trouble, let it be in my day,
 that my child may have peace."

Thomas Paine


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Caldarale, Charles R]

> From: Ivan Longhi [mailto:ivan.longhi@...]
> Subject: POST request hangs hup
>
> Is there a way to kill a single thread in tomcat whitout
> restarting it?

No.

> Is there a way to limit the max execution time for a single
> thread/request?

No.

You need to fix your webapp, since it has a serious bug resulting in an infinite loop.  Take some thread dumps a few seconds apart to find out where the thread is spending its time.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Christopher Schultz-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel,

On 7/2/2009 6:20 PM, Daniel Henrique Alves Lima wrote:
> I think could be anything: An infinite loop in your app, a database
> (or other resource) deadlock, any intensive db query (with a really
> long transaction timeout).

+1

Take a thread dump of your application to find out where the thread is
stuck (see
http://wiki.apache.org/tomcat/HowTo#head-a3243c1d640297a9f2aa9446cbed434efbd4d1d9
if you aren't sure how to take a thread dump). Several thread dumps a
few seconds apart is usually enough to give you a good picture of what's
happening.

Post them to the list if you are having a hard time determining what's
going on.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkpNbt0ACgkQ9CaO5/Lv0PArpQCgqIKN+mkCc+W8QCtru1S0Kjag
HrEAoJmO1k3SlqnuXBOpmRZwt8xFEJz5
=W5UC
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...