Flush for LockID took longer than 15000ms

View: New views
7 Messages — Rating Filter:   Alert me  

Flush for LockID took longer than 15000ms

by Thomas Chemineau :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


Hi,

Much better with a suitable mail subject.

I had some errors (warning, I don't know) with Terracotta. I use the 2.5.2
one, in a clustered environment (two terracotta servers in master/slave
mode). I had one error in Terracotta clients (use with Tomcat), that make
my application unavailable at all :

8<--------
2008-11-13 10:37:11,624 [DSO Lock Object.wait() timer] INFO
com.tc.object.lockmanager.api.ClientLockManager - ChannelID=[1]: Running
Lock GC...
2008-11-13 10:37:25,131 [http-8080-Processor18] INFO
com.tc.object.tx.RemoteTransactionManagerImpl - ChannelID=[1]: Flush for
LockID(@1499) took longer than: 15000ms. # Transactions not yet Acked = 1
8<--------

It seems it is a lock on an object. Can anyone help me to understand how
it occurs, and why ?

I know that you do not support versions before 2.7 for the community
edition. But I would like to understand what causes this error.

Thanks for the help,
Cheers,

Thomas.


_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users

Re: Flush for LockID took longer than 15000ms

by singh.raghvendra :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Its not exactly an error. This message is printed when the client tries
to take a lock and doesn't get it because its held somewhere else. Can
you check in your use case that you don't have any deadlock scenario?

I would still recommend you to upgrade it to 2.7 as it has a lot of bug
fixes in it.

-raghu

Thomas Chemineau wrote:

> Hi,
>
> Much better with a suitable mail subject.
>
> I had some errors (warning, I don't know) with Terracotta. I use the 2.5.2
> one, in a clustered environment (two terracotta servers in master/slave
> mode). I had one error in Terracotta clients (use with Tomcat), that make
> my application unavailable at all :
>
> 8<--------
> 2008-11-13 10:37:11,624 [DSO Lock Object.wait() timer] INFO
> com.tc.object.lockmanager.api.ClientLockManager - ChannelID=[1]: Running
> Lock GC...
> 2008-11-13 10:37:25,131 [http-8080-Processor18] INFO
> com.tc.object.tx.RemoteTransactionManagerImpl - ChannelID=[1]: Flush for
> LockID(@1499) took longer than: 15000ms. # Transactions not yet Acked = 1
> 8<--------
>
> It seems it is a lock on an object. Can anyone help me to understand how
> it occurs, and why ?
>
> I know that you do not support versions before 2.7 for the community
> edition. But I would like to understand what causes this error.
>
> Thanks for the help,
> Cheers,
>
> Thomas.
>
>
> _______________________________________________
> tc-users mailing list
> tc-users@...
> http://lists.terracotta.org/mailman/listinfo/tc-users
>  
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users

Re: Flush for LockID took longer than 15000ms

by singh.raghvendra :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Its not exactly an error. This message is printed when the client tries
to take a lock and doesn't get it because its held somewhere else. Can
you check in your use case that you don't have any deadlock scenario?

I would still recommend you to upgrade it to 2.7 as it has a lot of bug
fixes in it.

-raghu


Thomas Chemineau wrote:

> Hi,
>
> Much better with a suitable mail subject.
>
> I had some errors (warning, I don't know) with Terracotta. I use the 2.5.2
> one, in a clustered environment (two terracotta servers in master/slave
> mode). I had one error in Terracotta clients (use with Tomcat), that make
> my application unavailable at all :
>
> 8<--------
> 2008-11-13 10:37:11,624 [DSO Lock Object.wait() timer] INFO
> com.tc.object.lockmanager.api.ClientLockManager - ChannelID=[1]: Running
> Lock GC...
> 2008-11-13 10:37:25,131 [http-8080-Processor18] INFO
> com.tc.object.tx.RemoteTransactionManagerImpl - ChannelID=[1]: Flush for
> LockID(@1499) took longer than: 15000ms. # Transactions not yet Acked = 1
> 8<--------
>
> It seems it is a lock on an object. Can anyone help me to understand how
> it occurs, and why ?
>
> I know that you do not support versions before 2.7 for the community
> edition. But I would like to understand what causes this error.
>
> Thanks for the help,
> Cheers,
>
> Thomas.
>
>
> _______________________________________________
> tc-users mailing list
> tc-users@...
> http://lists.terracotta.org/mailman/listinfo/tc-users
>  
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users

Parent Message unknown Re: Flush for LockID took longer than 15000ms

by Thomas Chemineau :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


> Its not exactly an error. This message is printed when the client tries
> to take a lock and doesn't get it because its held somewhere else. Can
> you check in your use case that you don't have any deadlock scenario?

So, I understand that this is the layer I used between my application and
terracotta, or the application itself, that causes this "deadlock" error.

It is really annoying, because I had no logs. Can it be possible to turn
on debug mode on terracotta (server and client side) ?

To resolve this problem, I had to restart all servers of the cluster to
clear caches :/

> I would still recommend you to upgrade it to 2.7 as it has a lot of bug
> fixes in it.

I know :) But this is part of a another project. Here, I just want to
understand :)

Thank you,
Thomas.


_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users

Re: Flush for LockID took longer than 15000ms

by singh.raghvendra :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

There is a tag in dso section of the config named lock-debug, something
like below

<runtime-logging>
          <non-portable-dump>true</non-portable-dump>
          <lock-debug>false</lock-debug>
          <wait-notify-debug>false</wait-notify-debug>
          <distributed-method-debug>false</distributed-method-debug>
          <new-object-debug>false</new-object-debug>
</runtime-logging>

Set the lock-debug as true and restart your cluster. It will enable the
logging for locks but it will slow down the performance of your
application. Send us the logs if you again reproduce this case.

If you don't have any logs for your client then it is possible that your
clients are not at all connecting to the server array. Check in your
admin console that you can see all the clients connected to the server
array. If they are connected then turn on the lock profiling which can
help you in debugging the exact reason for dead lock.

Hope this helps

-raghu


Thomas Chemineau wrote:

>> Its not exactly an error. This message is printed when the client tries
>> to take a lock and doesn't get it because its held somewhere else. Can
>> you check in your use case that you don't have any deadlock scenario?
>>    
>
> So, I understand that this is the layer I used between my application and
> terracotta, or the application itself, that causes this "deadlock" error.
>
> It is really annoying, because I had no logs. Can it be possible to turn
> on debug mode on terracotta (server and client side) ?
>
> To resolve this problem, I had to restart all servers of the cluster to
> clear caches :/
>
>  
>> I would still recommend you to upgrade it to 2.7 as it has a lot of bug
>> fixes in it.
>>    
>
> I know :) But this is part of a another project. Here, I just want to
> understand :)
>
> Thank you,
> Thomas.
>
>
> _______________________________________________
> tc-users mailing list
> tc-users@...
> http://lists.terracotta.org/mailman/listinfo/tc-users
>  
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users

Re: Flush for LockID took longer than 15000ms

by Sergio Bossa :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Thu, Nov 13, 2008 at 11:19 AM, Thomas Chemineau
<tchemineau@...> wrote:

> It is really annoying, because I had no logs. Can it be possible to turn
> on debug mode on terracotta (server and client side) ?

I obviously agree with everything Raghvendra said.

Given that you're talking about Scarlet, I'd like to add that you can
automatically enable logging by setting the following property into
the Scarlet local.build.properties file (prior to building the whole
thing):
terracotta.runtime.logging=false

Finally, a clustered thread dump from the Terracotta admin console may
help, too.

Let us know.
Cheers,

Sergio B.

--
Sergio Bossa
Software Passionate, Java Technologies Specialist and Open Source Enthusiast.
Blog : http://sbtourist.blogspot.com
Sourcesense - making sense of Open Source : http://www.sourcesense.com
Pro-netics s.p.a. : http://www.pronetics.it
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users

Re: Flush for LockID took longer than 15000ms

by Taylor Gautier :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.
One thing about the lock-debug setting - you can turn that on and off dynamically at the console - it's not something you'd want on in a production scenario.
----- Original Message -----
From: "Sergio Bossa" <sergio.bossa@...>
To: tc-users@...
Sent: Thursday, November 13, 2008 2:59:22 AM GMT -08:00 US/Canada Pacific
Subject: Re: [tc-users] Flush for LockID took longer than 15000ms

On Thu, Nov 13, 2008 at 11:19 AM, Thomas Chemineau
<tchemineau@...> wrote:

> It is really annoying, because I had no logs. Can it be possible to turn
> on debug mode on terracotta (server and client side) ?

I obviously agree with everything Raghvendra said.

Given that you're talking about Scarlet, I'd like to add that you can
automatically enable logging by setting the following property into
the Scarlet local.build.properties file (prior to building the whole
thing):
terracotta.runtime.logging=false

Finally, a clustered thread dump from the Terracotta admin console may
help, too.

Let us know.
Cheers,

Sergio B.

--
Sergio Bossa
Software Passionate, Java Technologies Specialist and Open Source Enthusiast.
Blog : http://sbtourist.blogspot.com
Sourcesense - making sense of Open Source : http://www.sourcesense.com
Pro-netics s.p.a. : http://www.pronetics.it
_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users

_______________________________________________
tc-users mailing list
tc-users@...
http://lists.terracotta.org/mailman/listinfo/tc-users