errno 110 in mod_jk

[jgroups]

I am continously seeing the following even though all these nodes are
up and running and receving requests.

[Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
backend failed. Tomcat is probably not started or is listening on the
wrong port (errno=110)
[Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
failed.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[jgroups]

I turned on mod_jk debug and see:

[Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
[Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
[Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
lingering bytes
[Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
cping/cpong after connecting to the backend server failed (errno=110)
[Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
backend failed. Tomcat is probably not started or is listening on the
wrong port (errno=110)
[Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
tomcat failed (recoverable), because of error during request sending
(attempt=1)
[Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
before retrying
[Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
lingering bytes
[Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
cping/cpong after connecting to the backend server failed (errno=110)
[Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
backend failed. Tomcat is probably not started or is listening on the
wrong port (errno=110)

--
All nodes are up and have plenty of threads configured and free.

On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:

> I am continously seeing the following even though all these nodes are
> up and running and receving requests.
>
> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
> backend failed. Tomcat is probably not started or is listening on the
> wrong port (errno=110)
> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
> failed.
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Rainer Jung-3]

$ grep 110 /usr/include/*/errno.h

#define        ETIMEDOUT       110     /* Connection timed out */

On 07.11.2009 00:56, Mohit Anchlia wrote:
> I turned on mod_jk debug and see:

> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong

So your backend doesn't send the required cpong answer during the
timeout ou configured.

> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
> lingering bytes
> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
> cping/cpong after connecting to the backend server failed (errno=110)
> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
> backend failed. Tomcat is probably not started or is listening on the
> wrong port (errno=110)
> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
> tomcat failed (recoverable), because of error during request sending
> (attempt=1)
> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
> before retrying
> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
> lingering bytes
> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
> cping/cpong after connecting to the backend server failed (errno=110)
> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
> backend failed. Tomcat is probably not started or is listening on the
> wrong port (errno=110)
>
> --
> All nodes are up and have plenty of threads configured and free.

Yet they are still not responding in time.

You can verify using a apcket sniff.

> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>> I am continously seeing the following even though all these nodes are
>> up and running and receving requests.
>>
>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>> backend failed. Tomcat is probably not started or is listening on the
>> wrong port (errno=110)
>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>> failed.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[jgroups]

I am looking at the tcpdump but I don't see packet retransmits. What
should I expect to see in tcpdump? thanks for your help.

On Fri, Nov 6, 2009 at 4:05 PM, Rainer Jung <rainer.jung@...> wrote:

> $ grep 110 /usr/include/*/errno.h
>
> #define        ETIMEDOUT       110     /* Connection timed out */
>
> On 07.11.2009 00:56, Mohit Anchlia wrote:
>> I turned on mod_jk debug and see:
>
>> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
>> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
>
> So your backend doesn't send the required cpong answer during the
> timeout ou configured.
>
>> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
>> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>> lingering bytes
>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>> cping/cpong after connecting to the backend server failed (errno=110)
>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>> backend failed. Tomcat is probably not started or is listening on the
>> wrong port (errno=110)
>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
>> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
>> tomcat failed (recoverable), because of error during request sending
>> (attempt=1)
>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
>> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
>> before retrying
>> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>> lingering bytes
>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>> cping/cpong after connecting to the backend server failed (errno=110)
>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>> backend failed. Tomcat is probably not started or is listening on the
>> wrong port (errno=110)
>>
>> --
>> All nodes are up and have plenty of threads configured and free.
>
> Yet they are still not responding in time.
>
> You can verify using a apcket sniff.
>
>> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>> I am continously seeing the following even though all these nodes are
>>> up and running and receving requests.
>>>
>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>>> backend failed. Tomcat is probably not started or is listening on the
>>> wrong port (errno=110)
>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>>> failed.
>
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[awarnier]

Mohit Anchlia wrote:
> I am looking at the tcpdump but I don't see packet retransmits. What
> should I expect to see in tcpdump? thanks for your help.

Mohit,

have you checked that *all* the workers which your configure in
workers.properties really have a corresponding back-end Tomcat ?
Maybe you have one more in workers.properties than in reality ?


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[jgroups]

To narrow down I just left one node running and started test. When I
look at JkStatus worker I see even though all other nodes are down
it's still showing "OK" in the status. I am not sure why it would do
that. Could this be the reason why I am seeing slow response times in
mod-jk?

On Fri, Nov 6, 2009 at 6:30 PM, Mohit Anchlia <mohitanchlia@...> wrote:

> I am looking at the tcpdump but I don't see packet retransmits. What
> should I expect to see in tcpdump? thanks for your help.
>
> On Fri, Nov 6, 2009 at 4:05 PM, Rainer Jung <rainer.jung@...> wrote:
>> $ grep 110 /usr/include/*/errno.h
>>
>> #define        ETIMEDOUT       110     /* Connection timed out */
>>
>> On 07.11.2009 00:56, Mohit Anchlia wrote:
>>> I turned on mod_jk debug and see:
>>
>>> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
>>> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
>>
>> So your backend doesn't send the required cpong answer during the
>> timeout ou configured.
>>
>>> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
>>> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>> lingering bytes
>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>> cping/cpong after connecting to the backend server failed (errno=110)
>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>> backend failed. Tomcat is probably not started or is listening on the
>>> wrong port (errno=110)
>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
>>> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
>>> tomcat failed (recoverable), because of error during request sending
>>> (attempt=1)
>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
>>> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
>>> before retrying
>>> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>> lingering bytes
>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>> cping/cpong after connecting to the backend server failed (errno=110)
>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>> backend failed. Tomcat is probably not started or is listening on the
>>> wrong port (errno=110)
>>>
>>> --
>>> All nodes are up and have plenty of threads configured and free.
>>
>> Yet they are still not responding in time.
>>
>> You can verify using a apcket sniff.
>>
>>> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>> I am continously seeing the following even though all these nodes are
>>>> up and running and receving requests.
>>>>
>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>>>> backend failed. Tomcat is probably not started or is listening on the
>>>> wrong port (errno=110)
>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>>>> failed.
>>
>> Regards,
>>
>> Rainer
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@...
>> For additional commands, e-mail: users-help@...
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[jgroups]

I am also seeing that in Jboss (Back End) that there are lot of keep
alive sessions and I think that's the reason I am seeing the timeout.
As I add new web server it hits the 250 MaxThreads limit and then it
start to timeout. Those threads are there even though not doing any
work. I tried disabling keepalive in worker.properties but still those
threads are not being freed up.

Could someone please tell me why that might be happening?

On Sat, Nov 7, 2009 at 7:25 AM, Mohit Anchlia <mohitanchlia@...> wrote:

> To narrow down I just left one node running and started test. When I
> look at JkStatus worker I see even though all other nodes are down
> it's still showing "OK" in the status. I am not sure why it would do
> that. Could this be the reason why I am seeing slow response times in
> mod-jk?
>
> On Fri, Nov 6, 2009 at 6:30 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>> I am looking at the tcpdump but I don't see packet retransmits. What
>> should I expect to see in tcpdump? thanks for your help.
>>
>> On Fri, Nov 6, 2009 at 4:05 PM, Rainer Jung <rainer.jung@...> wrote:
>>> $ grep 110 /usr/include/*/errno.h
>>>
>>> #define        ETIMEDOUT       110     /* Connection timed out */
>>>
>>> On 07.11.2009 00:56, Mohit Anchlia wrote:
>>>> I turned on mod_jk debug and see:
>>>
>>>> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
>>>> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
>>>
>>> So your backend doesn't send the required cpong answer during the
>>> timeout ou configured.
>>>
>>>> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
>>>> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>> lingering bytes
>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>> backend failed. Tomcat is probably not started or is listening on the
>>>> wrong port (errno=110)
>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
>>>> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
>>>> tomcat failed (recoverable), because of error during request sending
>>>> (attempt=1)
>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
>>>> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
>>>> before retrying
>>>> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>> lingering bytes
>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>> backend failed. Tomcat is probably not started or is listening on the
>>>> wrong port (errno=110)
>>>>
>>>> --
>>>> All nodes are up and have plenty of threads configured and free.
>>>
>>> Yet they are still not responding in time.
>>>
>>> You can verify using a apcket sniff.
>>>
>>>> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>> I am continously seeing the following even though all these nodes are
>>>>> up and running and receving requests.
>>>>>
>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>> wrong port (errno=110)
>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>>>>> failed.
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@...
>>> For additional commands, e-mail: users-help@...
>>>
>>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[jgroups]

Could someone please help me understand this?

On Sat, Nov 7, 2009 at 8:50 AM, Mohit Anchlia <mohitanchlia@...> wrote:

> I am also seeing that in Jboss (Back End) that there are lot of keep
> alive sessions and I think that's the reason I am seeing the timeout.
> As I add new web server it hits the 250 MaxThreads limit and then it
> start to timeout. Those threads are there even though not doing any
> work. I tried disabling keepalive in worker.properties but still those
> threads are not being freed up.
>
> Could someone please tell me why that might be happening?
>
> On Sat, Nov 7, 2009 at 7:25 AM, Mohit Anchlia <mohitanchlia@...> wrote:
>> To narrow down I just left one node running and started test. When I
>> look at JkStatus worker I see even though all other nodes are down
>> it's still showing "OK" in the status. I am not sure why it would do
>> that. Could this be the reason why I am seeing slow response times in
>> mod-jk?
>>
>> On Fri, Nov 6, 2009 at 6:30 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>> I am looking at the tcpdump but I don't see packet retransmits. What
>>> should I expect to see in tcpdump? thanks for your help.
>>>
>>> On Fri, Nov 6, 2009 at 4:05 PM, Rainer Jung <rainer.jung@...> wrote:
>>>> $ grep 110 /usr/include/*/errno.h
>>>>
>>>> #define        ETIMEDOUT       110     /* Connection timed out */
>>>>
>>>> On 07.11.2009 00:56, Mohit Anchlia wrote:
>>>>> I turned on mod_jk debug and see:
>>>>
>>>>> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
>>>>> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
>>>>
>>>> So your backend doesn't send the required cpong answer during the
>>>> timeout ou configured.
>>>>
>>>>> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
>>>>> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>> lingering bytes
>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>> wrong port (errno=110)
>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
>>>>> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
>>>>> tomcat failed (recoverable), because of error during request sending
>>>>> (attempt=1)
>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
>>>>> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
>>>>> before retrying
>>>>> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>> lingering bytes
>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>> wrong port (errno=110)
>>>>>
>>>>> --
>>>>> All nodes are up and have plenty of threads configured and free.
>>>>
>>>> Yet they are still not responding in time.
>>>>
>>>> You can verify using a apcket sniff.
>>>>
>>>>> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>>> I am continously seeing the following even though all these nodes are
>>>>>> up and running and receving requests.
>>>>>>
>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>> wrong port (errno=110)
>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>>>>>> failed.
>>>>
>>>> Regards,
>>>>
>>>> Rainer
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@...
>>>> For additional commands, e-mail: users-help@...
>>>>
>>>>
>>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Rainer Jung-3]

On 08.11.2009 01:36, Mohit Anchlia wrote:
> Could someone please help me understand this?

What about adding connectionTimeout to the JBoss Connector element (and
keep its value in sync with connection_pool_timeout^for the ajp workers.
Note that the Connector attribute is in milliseconds, the worker
attribute in seconds.

Also make sure that you weren't hit by

https://jira.jboss.org/jira/browse/JBPAPP-366

Regards,

Rainer

> On Sat, Nov 7, 2009 at 8:50 AM, Mohit Anchlia <mohitanchlia@...> wrote:
>> I am also seeing that in Jboss (Back End) that there are lot of keep
>> alive sessions and I think that's the reason I am seeing the timeout.
>> As I add new web server it hits the 250 MaxThreads limit and then it
>> start to timeout. Those threads are there even though not doing any
>> work. I tried disabling keepalive in worker.properties but still those
>> threads are not being freed up.
>>
>> Could someone please tell me why that might be happening?
>>
>> On Sat, Nov 7, 2009 at 7:25 AM, Mohit Anchlia <mohitanchlia@...> wrote:
>>> To narrow down I just left one node running and started test. When I
>>> look at JkStatus worker I see even though all other nodes are down
>>> it's still showing "OK" in the status. I am not sure why it would do
>>> that. Could this be the reason why I am seeing slow response times in
>>> mod-jk?
>>>
>>> On Fri, Nov 6, 2009 at 6:30 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>> I am looking at the tcpdump but I don't see packet retransmits. What
>>>> should I expect to see in tcpdump? thanks for your help.
>>>>
>>>> On Fri, Nov 6, 2009 at 4:05 PM, Rainer Jung <rainer.jung@...> wrote:
>>>>> $ grep 110 /usr/include/*/errno.h
>>>>>
>>>>> #define        ETIMEDOUT       110     /* Connection timed out */
>>>>>
>>>>> On 07.11.2009 00:56, Mohit Anchlia wrote:
>>>>>> I turned on mod_jk debug and see:
>>>>>
>>>>>> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
>>>>>> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
>>>>>
>>>>> So your backend doesn't send the required cpong answer during the
>>>>> timeout ou configured.
>>>>>
>>>>>> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
>>>>>> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
>>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
>>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>>> lingering bytes
>>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
>>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>> wrong port (errno=110)
>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
>>>>>> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
>>>>>> tomcat failed (recoverable), because of error during request sending
>>>>>> (attempt=1)
>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
>>>>>> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
>>>>>> before retrying
>>>>>> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
>>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>>> lingering bytes
>>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>> wrong port (errno=110)
>>>>>>
>>>>>> --
>>>>>> All nodes are up and have plenty of threads configured and free.
>>>>>
>>>>> Yet they are still not responding in time.
>>>>>
>>>>> You can verify using a apcket sniff.
>>>>>
>>>>>> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>>>> I am continously seeing the following even though all these nodes are
>>>>>>> up and running and receving requests.
>>>>>>>
>>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>>> wrong port (errno=110)
>>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>>>>>>> failed.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Rainer
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@...
>>>>> For additional commands, e-mail: users-help@...

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[jgroups]

Thanks. Looks like we are hitting the same bug.

I am still looking for an answer to one of my questions that I posted earlier.

To narrow down I just left one node running and started the test. When I
look at JkStatus worker I see that even though all other nodes are down
it's still showing "OK" in the status for those workers. I am not sure
why it would do
that. I expect it to show the workers in ERR status.


Do you know the default timeout of connectionTimeout? I am getting
conflicting results. Some place it says 60000 and other places 600000.
Is there any disadvantage of setting this value too low?
On Sun, Nov 8, 2009 at 9:59 AM, Rainer Jung <rainer.jung@...> wrote:

> On 08.11.2009 01:36, Mohit Anchlia wrote:
>> Could someone please help me understand this?
>
> What about adding connectionTimeout to the JBoss Connector element (and
> keep its value in sync with connection_pool_timeout^for the ajp workers.
> Note that the Connector attribute is in milliseconds, the worker
> attribute in seconds.
>
> Also make sure that you weren't hit by
>
> https://jira.jboss.org/jira/browse/JBPAPP-366
>
> Regards,
>
> Rainer
>
>> On Sat, Nov 7, 2009 at 8:50 AM, Mohit Anchlia <mohitanchlia@...> wrote:
>>> I am also seeing that in Jboss (Back End) that there are lot of keep
>>> alive sessions and I think that's the reason I am seeing the timeout.
>>> As I add new web server it hits the 250 MaxThreads limit and then it
>>> start to timeout. Those threads are there even though not doing any
>>> work. I tried disabling keepalive in worker.properties but still those
>>> threads are not being freed up.
>>>
>>> Could someone please tell me why that might be happening?
>>>
>>> On Sat, Nov 7, 2009 at 7:25 AM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>> To narrow down I just left one node running and started test. When I
>>>> look at JkStatus worker I see even though all other nodes are down
>>>> it's still showing "OK" in the status. I am not sure why it would do
>>>> that. Could this be the reason why I am seeing slow response times in
>>>> mod-jk?
>>>>
>>>> On Fri, Nov 6, 2009 at 6:30 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>> I am looking at the tcpdump but I don't see packet retransmits. What
>>>>> should I expect to see in tcpdump? thanks for your help.
>>>>>
>>>>> On Fri, Nov 6, 2009 at 4:05 PM, Rainer Jung <rainer.jung@...> wrote:
>>>>>> $ grep 110 /usr/include/*/errno.h
>>>>>>
>>>>>> #define        ETIMEDOUT       110     /* Connection timed out */
>>>>>>
>>>>>> On 07.11.2009 00:56, Mohit Anchlia wrote:
>>>>>>> I turned on mod_jk debug and see:
>>>>>>
>>>>>>> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
>>>>>>> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
>>>>>>
>>>>>> So your backend doesn't send the required cpong answer during the
>>>>>> timeout ou configured.
>>>>>>
>>>>>>> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
>>>>>>> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
>>>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
>>>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>>>> lingering bytes
>>>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
>>>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
>>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>>> wrong port (errno=110)
>>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
>>>>>>> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
>>>>>>> tomcat failed (recoverable), because of error during request sending
>>>>>>> (attempt=1)
>>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
>>>>>>> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
>>>>>>> before retrying
>>>>>>> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
>>>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>>>> lingering bytes
>>>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>>> wrong port (errno=110)
>>>>>>>
>>>>>>> --
>>>>>>> All nodes are up and have plenty of threads configured and free.
>>>>>>
>>>>>> Yet they are still not responding in time.
>>>>>>
>>>>>> You can verify using a apcket sniff.
>>>>>>
>>>>>>> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>>>>> I am continously seeing the following even though all these nodes are
>>>>>>>> up and running and receving requests.
>>>>>>>>
>>>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>>>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>>>> wrong port (errno=110)
>>>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>>>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>>>>>>>> failed.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@...
>>>>>> For additional commands, e-mail: users-help@...
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Rainer Jung-3]

On 09.11.2009 03:13, Mohit Anchlia wrote:
> Thanks. Looks like we are hitting the same bug.
>
> I am still looking for an answer to one of my questions that I posted earlier.
>
> To narrow down I just left one node running and started the test. When I
> look at JkStatus worker I see that even though all other nodes are down
> it's still showing "OK" in the status for those workers. I am not sure
> why it would do
> that. I expect it to show the workers in ERR status.

No, there are two statuses: activation and state.
If you down a worker via activation, its activation will be displayed
either as disabled or stopped according to your config, but its runtime
status is OK (i.e. there is no error, because you put it out of order
administratively). It should be shown as OK/IDLE, i.e. we expect it to
work, but it hasn't been used for a long time, so we can't be sure.

status ERR is only shown, if a request detected the backend is not working.

> Do you know the default timeout of connectionTimeout? I am getting
> conflicting results. Some place it says 60000 and other places 600000.
> Is there any disadvantage of setting this value too low?

Default: no timeout.
Setting too low: the lower you set it, the more often connections will
be closed an later reopened. It is a question of how long a connection
should be reused. For server to server you want that to be typically > 1
minute. Why not resuing for a very long time, e.g. unlimited? Because in
case there is a firewall in between httpd and Tomcat it might drop idle
connections. Also you might want to free connections (and thus threads)
which were only necessary due to peak load or failover.

I usually suggest something between 1 and 10 minutes.

Regards,

Rainer

> On Sun, Nov 8, 2009 at 9:59 AM, Rainer Jung <rainer.jung@...> wrote:
>> On 08.11.2009 01:36, Mohit Anchlia wrote:
>>> Could someone please help me understand this?
>>
>> What about adding connectionTimeout to the JBoss Connector element (and
>> keep its value in sync with connection_pool_timeout^for the ajp workers.
>> Note that the Connector attribute is in milliseconds, the worker
>> attribute in seconds.
>>
>> Also make sure that you weren't hit by
>>
>> https://jira.jboss.org/jira/browse/JBPAPP-366
>>
>> Regards,
>>
>> Rainer
>>
>>> On Sat, Nov 7, 2009 at 8:50 AM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>> I am also seeing that in Jboss (Back End) that there are lot of keep
>>>> alive sessions and I think that's the reason I am seeing the timeout.
>>>> As I add new web server it hits the 250 MaxThreads limit and then it
>>>> start to timeout. Those threads are there even though not doing any
>>>> work. I tried disabling keepalive in worker.properties but still those
>>>> threads are not being freed up.
>>>>
>>>> Could someone please tell me why that might be happening?
>>>>
>>>> On Sat, Nov 7, 2009 at 7:25 AM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>> To narrow down I just left one node running and started test. When I
>>>>> look at JkStatus worker I see even though all other nodes are down
>>>>> it's still showing "OK" in the status. I am not sure why it would do
>>>>> that. Could this be the reason why I am seeing slow response times in
>>>>> mod-jk?
>>>>>
>>>>> On Fri, Nov 6, 2009 at 6:30 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>>> I am looking at the tcpdump but I don't see packet retransmits. What
>>>>>> should I expect to see in tcpdump? thanks for your help.
>>>>>>
>>>>>> On Fri, Nov 6, 2009 at 4:05 PM, Rainer Jung <rainer.jung@...> wrote:
>>>>>>> $ grep 110 /usr/include/*/errno.h
>>>>>>>
>>>>>>> #define        ETIMEDOUT       110     /* Connection timed out */
>>>>>>>
>>>>>>> On 07.11.2009 00:56, Mohit Anchlia wrote:
>>>>>>>> I turned on mod_jk debug and see:
>>>>>>>
>>>>>>>> [Fri Nov 06 15:48:31.190 2009] [31055:4143340384] [info]
>>>>>>>> ajp_handle_cping_cpong::jk_ajp_common.c (865): timeout in reply cpong
>>>>>>>
>>>>>>> So your backend doesn't send the required cpong answer during the
>>>>>>> timeout ou configured.
>>>>>>>
>>>>>>>> [Fri Nov 06 15:48:31.191 2009] [31055:4143340384] [debug]
>>>>>>>> jk_shutdown_socket::jk_connect.c (680): About to shutdown socket 14
>>>>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [debug]
>>>>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>>>>> lingering bytes
>>>>>>>> [Fri Nov 06 15:48:31.256 2009] [31044:4143340384] [error]
>>>>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [error]
>>>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>>>> wrong port (errno=110)
>>>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [info]
>>>>>>>> ajp_service::jk_ajp_common.c (2407): (eitappfe2) sending request to
>>>>>>>> tomcat failed (recoverable), because of error during request sending
>>>>>>>> (attempt=1)
>>>>>>>> [Fri Nov 06 15:48:31.257 2009] [31044:4143340384] [debug]
>>>>>>>> ajp_service::jk_ajp_common.c (2267): retry 1, sleeping for 100 ms
>>>>>>>> before retrying
>>>>>>>> [Fri Nov 06 15:48:31.314 2009] [31032:4143340384] [debug]
>>>>>>>> jk_shutdown_socket::jk_connect.c (731): Shutdown socket 14 and read 0
>>>>>>>> lingering bytes
>>>>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>>>>> ajp_connect_to_endpoint::jk_ajp_common.c (957): (eitappfe2)
>>>>>>>> cping/cpong after connecting to the backend server failed (errno=110)
>>>>>>>> [Fri Nov 06 15:48:31.315 2009] [31032:4143340384] [error]
>>>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe2) connecting to
>>>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>>>> wrong port (errno=110)
>>>>>>>>
>>>>>>>> --
>>>>>>>> All nodes are up and have plenty of threads configured and free.
>>>>>>>
>>>>>>> Yet they are still not responding in time.
>>>>>>>
>>>>>>> You can verify using a apcket sniff.
>>>>>>>
>>>>>>>> On Fri, Nov 6, 2009 at 3:47 PM, Mohit Anchlia <mohitanchlia@...> wrote:
>>>>>>>>> I am continously seeing the following even though all these nodes are
>>>>>>>>> up and running and receving requests.
>>>>>>>>>
>>>>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>>>>> ajp_send_request::jk_ajp_common.c (1467): (eitappfe3) connecting to
>>>>>>>>> backend failed. Tomcat is probably not started or is listening on the
>>>>>>>>> wrong port (errno=110)
>>>>>>>>> [Fri Nov 06 15:43:47.627 2009] [29200:4143332192] [error]
>>>>>>>>> ajp_service::jk_ajp_common.c (2426): (eitappfe3) connecting to tomcat
>>>>>>>>> failed.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...