Nabble - ActiveMQ - User

Karaf + ActiveMQ

2009-11-10T02:41:53Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi there,

I'm struggeling to deploy ActiveMQ on Karaf 1.0.
The problem I have is that the ActiveMQ bundles depend on the Geronimo
GShell which is not in the features listed here
mvn:org.apache.servicemix/apache-servicemix/4.0.0/xml/features.
And when I tried to add the required bundles manually I noticed that the
GSheel depends on com.google.code.sshd:sshd:bundle:0.1 instead of the
Mina SSHD implementation. This is all a bit hairy and hard to install.

Is there an up-to-date tutorial or documentation on how to do this?

Thanks in advance
Markus Wolf
- --
NMMN - New Media Markets & Networks GmbH
Langbehnstrasse 6, 22761 Hamburg
Geschäftsführung: Kfm. Michael Schütt
Finanzamt HH-Altona - UStID DE 812 699 852 - HRB 71102 Hamburg
HypoVereinsbank - BLZ 200 300 00 - Konto-Nr. 156 29 82

http://www.nmmn.com
Tel.: +49 40 284 118-0 - Fax: +49 40 284118-999
Softwareentwicklung LLynch: -720
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkr5Q3EACgkQDBHISU1oEKGNpQCgpG8wyZ0ZCXGMo2ya4bJGVCRc
PFEAoJ1rqRVHT7qSJjUIt9yY2pPLT/x4
=Pd9R
-----END PGP SIGNATURE-----

ANN: Habari ActiveMQ Client 2.0 released - Delphi 6-2010 and Free Pascal library

2009-11-10T01:35:14Z

November 10, 2009 - betasoft is pleased to announce release 2.0 of the
Delphi and Free Pascal library Habari ActiveMQ Client for the open
source Message Broker Apache ActiveMQ.

Habari ActiveMQ Client is a library for Delphi(tm) and Free Pascal which
provides easy access to Apache ActiveMQ, the most popular and powerful
open source Message Broker. With this library, Delphi developers can
build integrated solutions, connecting cross language clients and
protocols from Java(tm), C, C++, C#, Ruby, Perl, Python, and PHP, using
the peer-to-peer or the publish and subscribe communication model.

Habari ActiveMQ Client uses the Stomp message protocol and a plug-in
architecture for communication libraries and message transformers for
XML and JSON object serialization. It supports Apache ActiveMQ versions
4.0 to 5.3, Delphi 6 to 2010 and Free Pascal, and follows the
specification of the JMS API for Message Oriented Middleware.

Online Resources:

Habari JMS Client libraries home page:
http://www.mikejustin.com/habari.html

Habari ActiveMQ Client - Getting Started (PDF):
http://www.mikejustin.com/download/HabariActiveMQGettingStarted.pdf

Habari ActiveMQ Client - API Documentation
http://www.mikejustin.com/habari/2.0/docs/api/index.html

Habari ActiveMQ Client - Demo and free tools download:
http://www.mikejustin.com/download/HabariActiveMQ-demo.zip

About Apache ActiveMQ: Apache ActiveMQ is the most popular and powerful
open source Message Broker and Enterprise Integration Patterns provider.
Apache ActiveMQ is fast, supports many Cross Language Clients and
Protocols, comes with easy to use Enterprise Integration Patterns and
many advanced features while fully supporting JMS 1.1 and J2EE 1.4.
Read more about Apache ActiveMQ here: http://activemq.apache.org/

Best regards,

Michael Justin
SCJP, SCJA
betasoft - Software for Delphi™ and for the Java™ platform
http://www.mikejustin.com - http://www.betabeans.de

How to configure ActiveMQ externally with Jboss :

2009-11-10T01:30:33Z

Active MQ : apache-activemq-5.2.0
Jboss : jboss-5.1.0.GA

I am facing issues while configuring ActiveMQ with Jboss such that ActiveMQ should be externally running. It is working fine when i embed it inisde Jboss. But i donot want to embed inside JBoss.Even i donot want to deploy the Resource Adapter inside JBoss. I followed this link : http://activemq.apache.org/integrating-apache-activemq-with-jboss.html

Also how to configure ConnectionFactory, Queue, Topic inside Test-ds.xml which goes inside Jboss to map those with ActiveMQ resources ?

Could you please suggest what are the steps need to follow ?

Re: Message Head of line blocking

2009-11-10T00:50:46Z

Hi,

there's a default DLQ queue, but you can customize this if you need. You can
put a consumer there, or use web console to browse failed messages and do
appropriate actions manually.

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

On Mon, Nov 9, 2009 at 6:03 PM, David Erickson <derickso@...>wrote:

>
> Hi Dejan,
> Thanks for the reply. Do I need to explicitly declare the DLQ from within
> my Spring xml or is it created automatically? And I imagine I will need to
> put a consumer on the queue to do anything with it?
>
> Thanks,
> David
>
>
>
> Dejan Bosanac wrote:
> >
> > Hi David,
> >
> > it's all configured in your "redeliveryPolicy", where you instructed the
> > broker to try resending the message indefinitely.
> >
> > See http://activemq.apache.org/message-redelivery-and-dlq-handling.html
> > for
> > more information how redelivery works and how you can use "Dead letter
> > queues"
> >
> > Cheers
> > --
> > Dejan Bosanac - http://twitter.com/dejanb
> >
> > Open Source Integration - http://fusesource.com/
> > ActiveMQ in Action - http://www.manning.com/snyder/
> > Blog - http://www.nighttale.net
> >
> >
> > On Sat, Nov 7, 2009 at 7:35 PM, David Erickson
> > <derickso@...>wrote:
> >
> >>
> >> Hi All,
> >> I am using AMQ inside a Spring IoC container. It is a very simple setup
> >> with the following config:
> >>
> >> 
> >> <amq:broker useJmx="false" persistent="false">
> >> <amq:transportConnectors>
> >> <amq:transportConnector uri="tcp://localhost:0" />
> >> </amq:transportConnectors>
> >>
> >> </amq:broker>
> >>
> >> 
> >> <amq:queue id="mailDestination" physicalName="mailsender"/>
> >>
> >> 
> >> <amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost">
> >> <amq:redeliveryPolicy>
> >> <amq:redeliveryPolicy backOffMultiplier="2"
> >> useExponentialBackOff="true" initialRedeliveryDelay="1000"
> >> maximumRedeliveries="-1"/>
> >> </amq:redeliveryPolicy>
> >> </amq:connectionFactory>
> >>
> >> <bean id="myJmsTemplate"
> >> class="org.springframework.jms.core.JmsTemplate">
> >> <property name="connectionFactory">
> >> 
> >> <bean class="org.apache.activemq.pool.PooledConnectionFactory">
> >> <property name="connectionFactory">
> >> <ref local="jmsFactory"/>
> >> </property>
> >> </bean>
> >> </property>
> >> <property name="pubSubDomain">
> >> <value>false</value>
> >> </property>
> >> </bean>
> >>
> >> <bean id="mailContainer"
> >>
> class="org.springframework.jms.listener.DefaultMessageListenerContainer">
> >> <property name="connectionFactory" ref="jmsFactory"/>
> >> <property name="destination" ref="mailDestination"/>
> >> <property name="messageListener" ref="mailConsumer"/>
> >> <property name="sessionTransacted" value="true"/>
> >> </bean>
> >>
> >> Along with one corresponding producer and consumer.
> >>
> >> My question is this, I noticed that my consumer was having problems
> >> sending
> >> one of the emails in the queue, it was throwing an exception, being
> >> placed
> >> back in the queue, then the entire queue was waiting the backoff time
> for
> >> this message to attempt to be delivered again, over and over again,
> >> producing a head of line blocking problem (no messages can be delivered
> >> until the one with the error is delivered).
> >>
> >> Is there a solution for this? Can the message that failed be pushed to a
> >> re-delivery queue, or a troubled queue rather than the main queue? Or
> can
> >> the consumer walk through the queue looking for a message that is ready
> >> to
> >> be delivered rather than blocking on the front message that isn't set to
> >> be
> >> delivered for some long amount of time because of a previous failure and
> >> backoff time?
> >>
> >> Thanks!
> >> --
> >> View this message in context:
> >>
> http://old.nabble.com/Message-Head-of-line-blocking-tp26247283p26247283.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > -----
> > Dejan Bosanac
> >
> > Open Source Integration - http://fusesource.com/
> > ActiveMQ in Action - http://www.manning.com/snyder/
> > Blog - http://www.nighttale.net
> >
>
> --
> View this message in context:
> http://old.nabble.com/Message-Head-of-line-blocking-tp26247283p26269877.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

Dejan Bosanac

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

Re: Out of Memory on 5.3

2009-11-10T00:06:17Z

in org.apache.activemq.broker.region ,why so many invoke asyncWakeup(),
what is the method:iterate() doing?

afei wrote:

in addition,another problem of OOM.

Gary Tully wrote:

fyi: you can disable periodic message expiry processing using a
destination policy entry that sets expireMessagesPeriod = 0

2009/11/9 afei <afei1689@126.com>:
>
>
> when a large of messages in queue,and no consumer or the consumer is very
> slow, the OOM problem occur, because :
> in org.apache.activemq.broker.region.Queue,the 588 line is :
> doBrowse(true, browsedMessages, this.getMaxExpirePageSize());
> ,transform to :
> doBrowse(false, browsedMessages, this.getMaxExpirePageSize());
> is ok.
>
>
> Dejan Bosanac wrote:
>>
>> Hi Mitch,
>>
>> yeah, I said in thread I was referring to, that it is working with
>> "regular"
>> stomp connector. I started investigating AMQ-2440 patch the other day,
>> should be have something soon.
>>
>> Cheers
>> --
>> Dejan Bosanac - http://twitter.com/dejanb
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
>>
>> On Wed, Oct 28, 2009 at 6:18 PM, Mitch Granger
>> <mitch.granger@sophos.com>wrote:
>>
>>> So we turned off stomp+nio and went back to plain old stomp and so far
>>> it's
>>> working fine. New(IO) isn't always better, I guess :-)
>>>
>>> Seems like maybe it's this issue ->
>>> https://issues.apache.org/activemq/browse/AMQ-2440
>>>
>>>
>>> afei wrote:
>>>
>>>> i have same problem
>>>>
>>>> http://www.nabble.com/file/p26093204/aaaaaa.jpg aaaaaa.jpg
>>>>
>>>>
>>>> themitchy wrote:
>>>>
>>>>> This is what we've done to tune so far:
>>>>>
>>>>> - UseDedicatedTaskRunner=false
>>>>> - flow control is off
>>>>> - stomp transport uses transport.closeAsync=false
>>>>>
>>>>> I agree that it is because of the high number of open/close connections
>>>>> from Stomp. When we monitor through JConsole we can see more threads
>>>>> starting up for each new connection. The problem is that these threads
>>>>> don't get let go. Even though the stomp clients are disconnecting the
>>>>> number of threads that get released is less than the number created.
>>>>> So
>>>>> the thread count goes up and up until it fails. All of the above
>>>>> settings/tuning only delay when it will hit the wall.
>>>>>
>>>>> Dejan Bosanac wrote:
>>>>>
>>>>>> Hi Mitch,
>>>>>>
>>>>>> I think the root cause of this problem is that you probably have Stomp
>>>>>> clients that open/close connection at a high rate. I simulated this
>>>>>> problem
>>>>>> on OSX with a StompLoadTest (
>>>>>>
>>>>>> http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/transport/stomp/StompLoadTest.java?view=log
>>>>>> ),
>>>>>> while trying to reproduce "too many open files" problem. You can find
>>>>>> some
>>>>>> of my findings (and workaround) in this thread.
>>>>>>
>>>>>>
>>>>>> http://www.nabble.com/%22too-many-open-files%22-error-with-5.3-and-Stomp-tt25888831.html#a26010080
>>>>>>
>>>>>> (BTW. it is producing "too many open files" problem on linux)
>>>>>> Basically,
>>>>>> the
>>>>>> problem with stomp is that every send is done in separate connection
>>>>>> and
>>>>>> thus considered to be a new producer for every message. So when
>>>>>> producer
>>>>>> flow control is hit, the producers are piling up and probably not
>>>>>> releasing
>>>>>> connections. Thus you can observe large number of tcp connections on
>>>>>> the
>>>>>> system in state TIME_WAIT (and TIME_CLOSE), which causes that the
>>>>>> system
>>>>>> limit is hit at one point. In the above thread, you can find a
>>>>>> workaround
>>>>>> that worked for me for that test. I started investigating this more
>>>>>> and
>>>>>> hopefully I'll have some more findings in the near future.
>>>>>>
>>>>>> Cheers
>>>>>> --
>>>>>> Dejan Bosanac - http://twitter.com/dejanb
>>>>>>
>>>>>> Open Source Integration - http://fusesource.com/
>>>>>> ActiveMQ in Action - http://www.manning.com/snyder/
>>>>>> Blog - http://www.nighttale.net
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 27, 2009 at 1:07 AM, Mitch Granger
>>>>>> <mitch.granger@sophos.com>wrote:
>>>>>>
>>>>>> Update: We've [nearly] proven that this only happens with AMQ running
>>>>>>> on
>>>>>>> openVZ. What exactly is causing it, we're still not sure. After
>>>>>>> memoryUsage is met, the number of threads skyrockets until we get
>>>>>>> OutOfMemoryError.
>>>>>>>
>>>>>>> It works just fine on regular hardware; We're going to try VMWare
>>>>>>> tomorrow.
>>>>>>>
>>>>>>> One thing really worth mentioning is that by using the fileCursor we
>>>>>>> actually started seeing it use the Temp Store. When reading about
>>>>>>> systemUsage it is NOT intuitive that the Temp Store does not come
>>>>>>> into
>>>>>>> play
>>>>>>> with the default cursor. Anyone keeping a significant volume of
>>>>>>> messages on
>>>>>>> their queues should be well served by changing the cursor.
>>>>>>>
>>>>>>>
>>>>>>> Mitch Granger wrote:
>>>>>>>
>>>>>>> Config is attached. We have also tried the activemq-scalability.xml
>>>>>>>> with
>>>>>>>> the only change being adding a stomp connector.
>>>>>>>>
>>>>>>>> Once we hit the memoryUsage limit we can [sometimes] connect new
>>>>>>>> consumers
>>>>>>>> but nothing comes back after we send the SUBSCRIBE frame.
>>>>>>>>
>>>>>>>> I expect sending to fail when we hit this limit but if we can't
>>>>>>>> subscribe
>>>>>>>> there's no chance of recovering from this state.
>>>>>>>>
>>>>>>>> Rob Davies wrote:
>>>>>>>>
>>>>>>>> On 26 Oct 2009, at 17:38, themitchy wrote:
>>>>>>>>>
>>>>>>>>> We're using only persistent messages and heap size is set to 2GB
>>>>>>>>> yet
>>>>>>>>> we
>>>>>>>>>
>>>>>>>>>> hit
>>>>>>>>>> the memoryUsage limit quite quickly (system usage config below).
>>>>>>>>>> This
>>>>>>>>>> is
>>>>>>>>>> followed by "java.lang.OutOfMemoryError: unable to create new
>>>>>>>>>> native
>>>>>>>>>> thread"
>>>>>>>>>> as the process quickly reaches the 2GB of heap we gave it. How
>>>>>>>>>> are
>>>>>>>>>> we
>>>>>>>>>> getting to that point with the memoryUsage limit set far below it?
>>>>>>>>>>
>>>>>>>>>> Is there no way to get AMQ to gracefully limit it's memory usage?
>>>>>>>>>>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <memoryUsage>
>>>>>>>>>> <memoryUsage limit="256 mb"/>
>>>>>>>>>> </memoryUsage>
>>>>>>>>>> <storeUsage>
>>>>>>>>>> <storeUsage limit="60 gb" name="foo"/>
>>>>>>>>>> </storeUsage>
>>>>>>>>>> <tempUsage>
>>>>>>>>>> <tempUsage limit="60 gb"/>
>>>>>>>>>> </tempUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Out-of-Memory-on-5.3-tp26064098p26064098.html
>>>>>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Can you send the rest of your config ?
>>>>>>>>>
>>>>>>>>> Rob Davies
>>>>>>>>> http://twitter.com/rajdavies
>>>>>>>>> I work here: http://fusesource.com
>>>>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>
>>
>> -----
>> Dejan Bosanac
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
> http://old.nabble.com/file/p26264779/dump.jpg
> --
> View this message in context: http://old.nabble.com/Out-of-Memory-on-5.3-tp26064098p26264779.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Re: Out of Memory on 5.3

2009-11-09T20:45:14Z

in addition,another problem of OOM.

Gary Tully wrote:

fyi: you can disable periodic message expiry processing using a
destination policy entry that sets expireMessagesPeriod = 0

2009/11/9 afei <afei1689@126.com>:
>
>
> when a large of messages in queue,and no consumer or the consumer is very
> slow, the OOM problem occur, because :
> in org.apache.activemq.broker.region.Queue,the 588 line is :
> doBrowse(true, browsedMessages, this.getMaxExpirePageSize());
> ,transform to :
> doBrowse(false, browsedMessages, this.getMaxExpirePageSize());
> is ok.
>
>
> Dejan Bosanac wrote:
>>
>> Hi Mitch,
>>
>> yeah, I said in thread I was referring to, that it is working with
>> "regular"
>> stomp connector. I started investigating AMQ-2440 patch the other day,
>> should be have something soon.
>>
>> Cheers
>> --
>> Dejan Bosanac - http://twitter.com/dejanb
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
>>
>> On Wed, Oct 28, 2009 at 6:18 PM, Mitch Granger
>> <mitch.granger@sophos.com>wrote:
>>
>>> So we turned off stomp+nio and went back to plain old stomp and so far
>>> it's
>>> working fine. New(IO) isn't always better, I guess :-)
>>>
>>> Seems like maybe it's this issue ->
>>> https://issues.apache.org/activemq/browse/AMQ-2440
>>>
>>>
>>> afei wrote:
>>>
>>>> i have same problem
>>>>
>>>> http://www.nabble.com/file/p26093204/aaaaaa.jpg aaaaaa.jpg
>>>>
>>>>
>>>> themitchy wrote:
>>>>
>>>>> This is what we've done to tune so far:
>>>>>
>>>>> - UseDedicatedTaskRunner=false
>>>>> - flow control is off
>>>>> - stomp transport uses transport.closeAsync=false
>>>>>
>>>>> I agree that it is because of the high number of open/close connections
>>>>> from Stomp. When we monitor through JConsole we can see more threads
>>>>> starting up for each new connection. The problem is that these threads
>>>>> don't get let go. Even though the stomp clients are disconnecting the
>>>>> number of threads that get released is less than the number created.
>>>>> So
>>>>> the thread count goes up and up until it fails. All of the above
>>>>> settings/tuning only delay when it will hit the wall.
>>>>>
>>>>> Dejan Bosanac wrote:
>>>>>
>>>>>> Hi Mitch,
>>>>>>
>>>>>> I think the root cause of this problem is that you probably have Stomp
>>>>>> clients that open/close connection at a high rate. I simulated this
>>>>>> problem
>>>>>> on OSX with a StompLoadTest (
>>>>>>
>>>>>> http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/transport/stomp/StompLoadTest.java?view=log
>>>>>> ),
>>>>>> while trying to reproduce "too many open files" problem. You can find
>>>>>> some
>>>>>> of my findings (and workaround) in this thread.
>>>>>>
>>>>>>
>>>>>> http://www.nabble.com/%22too-many-open-files%22-error-with-5.3-and-Stomp-tt25888831.html#a26010080
>>>>>>
>>>>>> (BTW. it is producing "too many open files" problem on linux)
>>>>>> Basically,
>>>>>> the
>>>>>> problem with stomp is that every send is done in separate connection
>>>>>> and
>>>>>> thus considered to be a new producer for every message. So when
>>>>>> producer
>>>>>> flow control is hit, the producers are piling up and probably not
>>>>>> releasing
>>>>>> connections. Thus you can observe large number of tcp connections on
>>>>>> the
>>>>>> system in state TIME_WAIT (and TIME_CLOSE), which causes that the
>>>>>> system
>>>>>> limit is hit at one point. In the above thread, you can find a
>>>>>> workaround
>>>>>> that worked for me for that test. I started investigating this more
>>>>>> and
>>>>>> hopefully I'll have some more findings in the near future.
>>>>>>
>>>>>> Cheers
>>>>>> --
>>>>>> Dejan Bosanac - http://twitter.com/dejanb
>>>>>>
>>>>>> Open Source Integration - http://fusesource.com/
>>>>>> ActiveMQ in Action - http://www.manning.com/snyder/
>>>>>> Blog - http://www.nighttale.net
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 27, 2009 at 1:07 AM, Mitch Granger
>>>>>> <mitch.granger@sophos.com>wrote:
>>>>>>
>>>>>> Update: We've [nearly] proven that this only happens with AMQ running
>>>>>>> on
>>>>>>> openVZ. What exactly is causing it, we're still not sure. After
>>>>>>> memoryUsage is met, the number of threads skyrockets until we get
>>>>>>> OutOfMemoryError.
>>>>>>>
>>>>>>> It works just fine on regular hardware; We're going to try VMWare
>>>>>>> tomorrow.
>>>>>>>
>>>>>>> One thing really worth mentioning is that by using the fileCursor we
>>>>>>> actually started seeing it use the Temp Store. When reading about
>>>>>>> systemUsage it is NOT intuitive that the Temp Store does not come
>>>>>>> into
>>>>>>> play
>>>>>>> with the default cursor. Anyone keeping a significant volume of
>>>>>>> messages on
>>>>>>> their queues should be well served by changing the cursor.
>>>>>>>
>>>>>>>
>>>>>>> Mitch Granger wrote:
>>>>>>>
>>>>>>> Config is attached. We have also tried the activemq-scalability.xml
>>>>>>>> with
>>>>>>>> the only change being adding a stomp connector.
>>>>>>>>
>>>>>>>> Once we hit the memoryUsage limit we can [sometimes] connect new
>>>>>>>> consumers
>>>>>>>> but nothing comes back after we send the SUBSCRIBE frame.
>>>>>>>>
>>>>>>>> I expect sending to fail when we hit this limit but if we can't
>>>>>>>> subscribe
>>>>>>>> there's no chance of recovering from this state.
>>>>>>>>
>>>>>>>> Rob Davies wrote:
>>>>>>>>
>>>>>>>> On 26 Oct 2009, at 17:38, themitchy wrote:
>>>>>>>>>
>>>>>>>>> We're using only persistent messages and heap size is set to 2GB
>>>>>>>>> yet
>>>>>>>>> we
>>>>>>>>>
>>>>>>>>>> hit
>>>>>>>>>> the memoryUsage limit quite quickly (system usage config below).
>>>>>>>>>> This
>>>>>>>>>> is
>>>>>>>>>> followed by "java.lang.OutOfMemoryError: unable to create new
>>>>>>>>>> native
>>>>>>>>>> thread"
>>>>>>>>>> as the process quickly reaches the 2GB of heap we gave it. How
>>>>>>>>>> are
>>>>>>>>>> we
>>>>>>>>>> getting to that point with the memoryUsage limit set far below it?
>>>>>>>>>>
>>>>>>>>>> Is there no way to get AMQ to gracefully limit it's memory usage?
>>>>>>>>>>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <memoryUsage>
>>>>>>>>>> <memoryUsage limit="256 mb"/>
>>>>>>>>>> </memoryUsage>
>>>>>>>>>> <storeUsage>
>>>>>>>>>> <storeUsage limit="60 gb" name="foo"/>
>>>>>>>>>> </storeUsage>
>>>>>>>>>> <tempUsage>
>>>>>>>>>> <tempUsage limit="60 gb"/>
>>>>>>>>>> </tempUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Out-of-Memory-on-5.3-tp26064098p26064098.html
>>>>>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Can you send the rest of your config ?
>>>>>>>>>
>>>>>>>>> Rob Davies
>>>>>>>>> http://twitter.com/rajdavies
>>>>>>>>> I work here: http://fusesource.com
>>>>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>
>>
>> -----
>> Dejan Bosanac
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
> http://old.nabble.com/file/p26264779/dump.jpg
> --
> View this message in context: http://old.nabble.com/Out-of-Memory-on-5.3-tp26064098p26264779.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Re: Out of Memory on 5.3

2009-11-09T20:37:40Z

i test r831258,the problem still exist

Gary Tully wrote:

fyi: you can disable periodic message expiry processing using a
destination policy entry that sets expireMessagesPeriod = 0

2009/11/9 afei <afei1689@126.com>:
>
>
> when a large of messages in queue,and no consumer or the consumer is very
> slow, the OOM problem occur, because :
> in org.apache.activemq.broker.region.Queue,the 588 line is :
> doBrowse(true, browsedMessages, this.getMaxExpirePageSize());
> ,transform to :
> doBrowse(false, browsedMessages, this.getMaxExpirePageSize());
> is ok.
>
>
> Dejan Bosanac wrote:
>>
>> Hi Mitch,
>>
>> yeah, I said in thread I was referring to, that it is working with
>> "regular"
>> stomp connector. I started investigating AMQ-2440 patch the other day,
>> should be have something soon.
>>
>> Cheers
>> --
>> Dejan Bosanac - http://twitter.com/dejanb
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
>>
>> On Wed, Oct 28, 2009 at 6:18 PM, Mitch Granger
>> <mitch.granger@sophos.com>wrote:
>>
>>> So we turned off stomp+nio and went back to plain old stomp and so far
>>> it's
>>> working fine. New(IO) isn't always better, I guess :-)
>>>
>>> Seems like maybe it's this issue ->
>>> https://issues.apache.org/activemq/browse/AMQ-2440
>>>
>>>
>>> afei wrote:
>>>
>>>> i have same problem
>>>>
>>>> http://www.nabble.com/file/p26093204/aaaaaa.jpg aaaaaa.jpg
>>>>
>>>>
>>>> themitchy wrote:
>>>>
>>>>> This is what we've done to tune so far:
>>>>>
>>>>> - UseDedicatedTaskRunner=false
>>>>> - flow control is off
>>>>> - stomp transport uses transport.closeAsync=false
>>>>>
>>>>> I agree that it is because of the high number of open/close connections
>>>>> from Stomp. When we monitor through JConsole we can see more threads
>>>>> starting up for each new connection. The problem is that these threads
>>>>> don't get let go. Even though the stomp clients are disconnecting the
>>>>> number of threads that get released is less than the number created.
>>>>> So
>>>>> the thread count goes up and up until it fails. All of the above
>>>>> settings/tuning only delay when it will hit the wall.
>>>>>
>>>>> Dejan Bosanac wrote:
>>>>>
>>>>>> Hi Mitch,
>>>>>>
>>>>>> I think the root cause of this problem is that you probably have Stomp
>>>>>> clients that open/close connection at a high rate. I simulated this
>>>>>> problem
>>>>>> on OSX with a StompLoadTest (
>>>>>>
>>>>>> http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/transport/stomp/StompLoadTest.java?view=log
>>>>>> ),
>>>>>> while trying to reproduce "too many open files" problem. You can find
>>>>>> some
>>>>>> of my findings (and workaround) in this thread.
>>>>>>
>>>>>>
>>>>>> http://www.nabble.com/%22too-many-open-files%22-error-with-5.3-and-Stomp-tt25888831.html#a26010080
>>>>>>
>>>>>> (BTW. it is producing "too many open files" problem on linux)
>>>>>> Basically,
>>>>>> the
>>>>>> problem with stomp is that every send is done in separate connection
>>>>>> and
>>>>>> thus considered to be a new producer for every message. So when
>>>>>> producer
>>>>>> flow control is hit, the producers are piling up and probably not
>>>>>> releasing
>>>>>> connections. Thus you can observe large number of tcp connections on
>>>>>> the
>>>>>> system in state TIME_WAIT (and TIME_CLOSE), which causes that the
>>>>>> system
>>>>>> limit is hit at one point. In the above thread, you can find a
>>>>>> workaround
>>>>>> that worked for me for that test. I started investigating this more
>>>>>> and
>>>>>> hopefully I'll have some more findings in the near future.
>>>>>>
>>>>>> Cheers
>>>>>> --
>>>>>> Dejan Bosanac - http://twitter.com/dejanb
>>>>>>
>>>>>> Open Source Integration - http://fusesource.com/
>>>>>> ActiveMQ in Action - http://www.manning.com/snyder/
>>>>>> Blog - http://www.nighttale.net
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 27, 2009 at 1:07 AM, Mitch Granger
>>>>>> <mitch.granger@sophos.com>wrote:
>>>>>>
>>>>>> Update: We've [nearly] proven that this only happens with AMQ running
>>>>>>> on
>>>>>>> openVZ. What exactly is causing it, we're still not sure. After
>>>>>>> memoryUsage is met, the number of threads skyrockets until we get
>>>>>>> OutOfMemoryError.
>>>>>>>
>>>>>>> It works just fine on regular hardware; We're going to try VMWare
>>>>>>> tomorrow.
>>>>>>>
>>>>>>> One thing really worth mentioning is that by using the fileCursor we
>>>>>>> actually started seeing it use the Temp Store. When reading about
>>>>>>> systemUsage it is NOT intuitive that the Temp Store does not come
>>>>>>> into
>>>>>>> play
>>>>>>> with the default cursor. Anyone keeping a significant volume of
>>>>>>> messages on
>>>>>>> their queues should be well served by changing the cursor.
>>>>>>>
>>>>>>>
>>>>>>> Mitch Granger wrote:
>>>>>>>
>>>>>>> Config is attached. We have also tried the activemq-scalability.xml
>>>>>>>> with
>>>>>>>> the only change being adding a stomp connector.
>>>>>>>>
>>>>>>>> Once we hit the memoryUsage limit we can [sometimes] connect new
>>>>>>>> consumers
>>>>>>>> but nothing comes back after we send the SUBSCRIBE frame.
>>>>>>>>
>>>>>>>> I expect sending to fail when we hit this limit but if we can't
>>>>>>>> subscribe
>>>>>>>> there's no chance of recovering from this state.
>>>>>>>>
>>>>>>>> Rob Davies wrote:
>>>>>>>>
>>>>>>>> On 26 Oct 2009, at 17:38, themitchy wrote:
>>>>>>>>>
>>>>>>>>> We're using only persistent messages and heap size is set to 2GB
>>>>>>>>> yet
>>>>>>>>> we
>>>>>>>>>
>>>>>>>>>> hit
>>>>>>>>>> the memoryUsage limit quite quickly (system usage config below).
>>>>>>>>>> This
>>>>>>>>>> is
>>>>>>>>>> followed by "java.lang.OutOfMemoryError: unable to create new
>>>>>>>>>> native
>>>>>>>>>> thread"
>>>>>>>>>> as the process quickly reaches the 2GB of heap we gave it. How
>>>>>>>>>> are
>>>>>>>>>> we
>>>>>>>>>> getting to that point with the memoryUsage limit set far below it?
>>>>>>>>>>
>>>>>>>>>> Is there no way to get AMQ to gracefully limit it's memory usage?
>>>>>>>>>>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <memoryUsage>
>>>>>>>>>> <memoryUsage limit="256 mb"/>
>>>>>>>>>> </memoryUsage>
>>>>>>>>>> <storeUsage>
>>>>>>>>>> <storeUsage limit="60 gb" name="foo"/>
>>>>>>>>>> </storeUsage>
>>>>>>>>>> <tempUsage>
>>>>>>>>>> <tempUsage limit="60 gb"/>
>>>>>>>>>> </tempUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Out-of-Memory-on-5.3-tp26064098p26064098.html
>>>>>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Can you send the rest of your config ?
>>>>>>>>>
>>>>>>>>> Rob Davies
>>>>>>>>> http://twitter.com/rajdavies
>>>>>>>>> I work here: http://fusesource.com
>>>>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>
>>
>> -----
>> Dejan Bosanac
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
> http://old.nabble.com/file/p26264779/dump.jpg
> --
> View this message in context: http://old.nabble.com/Out-of-Memory-on-5.3-tp26064098p26264779.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Re: activemq bridge

2009-11-09T19:56:56Z

Forgot to mention that activemq is on another server which is independent from ServerA and ServerB.
Both ServerA and ServerB use jboss messaging.

jeff yung wrote:

Is it possible to make use of activemq bridge to forward the messages from a remote queue on serverA to another remote queue on ServerB?
If yes, how should I configure the activemq? Thanks.

activemq bridge

2009-11-09T19:31:22Z

Is it possible to make use of activemq bridge to forward the messages from a remote queue on serverA to another remote queue on ServerB?
If yes, how should I configure the activemq? Thanks.

NegativeArraySizeException after ActiveMQ 5.3 restart

2009-11-09T12:14:58Z

I am using ActiveMQ 5.3 with persistent messages through a MySQL database and recently had a repeating error occur in the wrapper.log file. When I stopped ActiveMQ, there were a few messages in the database. When ActiveMQ was started I noticed the error below occur once the number of messages in the database went below the value when ActiveMQ was started. I was just wondering if this was normal. I restarted ActiveMQ again and before I started the service I made sure there were no messages in the ACTIVEMQ_MSGS table. Since the restart, I haven't see the error in the wrapper.log file. Thank you.

java.lang.NegativeArraySizeException
at org.apache.activemq.openwire.v5.BaseDataStreamMarshaller.looseUnmarshalByteSequence(BaseDataStreamMarshaller.java:639)
at org.apache.activemq.openwire.v5.MessageMarshaller.looseUnmarshal(MessageMarshaller.java:229)
at org.apache.activemq.openwire.v5.ActiveMQMessageMarshaller.looseUnmarshal(ActiveMQMessageMarshaller.java:101)
at org.apache.activemq.openwire.v5.ActiveMQObjectMessageMarshaller.looseUnmarshal(ActiveMQObjectMessageMarshaller.java:101)
at org.apache.activemq.openwire.OpenWireFormat.doUnmarshal(OpenWireFormat.java:368)
at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:207)
at org.apache.activemq.store.jdbc.JDBCMessageStore$2.recoverMessage(JDBCMessageStore.java:218)
at org.apache.activemq.store.jdbc.adapter.DefaultJDBCAdapter.doRecoverNextMessages(DefaultJDBCAdapter.java:750)
at org.apache.activemq.store.jdbc.JDBCMessageStore.recoverNextMessages(JDBCMessageStore.java:214)
at org.apache.activemq.store.ProxyMessageStore.recoverNextMessages(ProxyMessageStore.java:87)
at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:81)
at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:227)
at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.hasNext(AbstractStoreCursor.java:134)
at org.apache.activemq.broker.region.cursors.StoreQueueCursor.hasNext(StoreQueueCursor.java:131)
at org.apache.activemq.broker.region.Queue.doPageIn(Queue.java:1364)
at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1503)
at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1178)
at org.apache.activemq.thread.DeterministicTaskRunner.runTask(DeterministicTaskRunner.java:84)
at org.apache.activemq.thread.DeterministicTaskRunner$1.run(DeterministicTaskRunner.java:41)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

ActiveMQ Consumer / Producer Connection Listener

2009-11-09T11:13:05Z

I can't seem to find a way to listen for new producer and consumer connections (or connection interrupts) in ActiveMQ (Java Version). I want to be able to tell the consumers (or they can find out themselves) that the producer's connection dropped. The other way around (the producer finding out that a certain consumer disconnected) is also required.

I'd appreciate if someone could help me.

Re: Message delivering.

2009-11-09T10:09:39Z

Hello again!

I resolved message redelivering problem, by following:
add to subscriber code:

IMessageConsumer consumer = session.CreateDurableConsumer(new Apache.NMS.ActiveMQ.Commands.ActiveMQTopic("testing?consumer.retroactive=false"), "testing listener", null, false);

and to config file:

<subscriptionRecoveryPolicy>
<timedSubscriptionRecoveryPolicy recoverDuration="120000" />
</subscriptionRecoveryPolicy>

but there is another problem

I'm using all this in a failover mode, using filesharing (see my previous message) and when I run pulisher for some time and then close him and kill current broker, it switches to another broker (failover working), I run subscriber and it doesn't get messages. If I dont' kill broker all works good. Any ideas?

2 duplex network connections between 2 brokers.

2009-11-09T09:59:49Z

Hi

For administrative reasons, I have 2 brokers that are connected to each others twice in the two ways

First broker
TransportConnector multicast group A
NetworkConnector multicast group B

Second broker
TransportConnector multicast group B
NetworkConnector multicast group A

I know that it can seem better only to have one exchange duplex connection in such a configuration but it is difficult because I want to be able to split my data flows by their nature (applicative flows or administrative flows) and associate a different network of broker.

I have two environments with two different versions of ActiveMQ 5.2.
On the first one, no error, my brokers are connected twice.
On the second one, I have an Async Error : already connected.

I want to know if my environment with no error has an ActiveMQ behaviour that is the one that is waited for or if the normal behaviour is to have an error.

I don't find clearly the answer by looking for this in the forum.

Thank you in advance.
Eric-AWL

Re: Message Head of line blocking

2009-11-09T09:03:41Z

Hi Dejan,
Thanks for the reply. Do I need to explicitly declare the DLQ from within my Spring xml or is it created automatically? And I imagine I will need to put a consumer on the queue to do anything with it?

Thanks,
David

Dejan Bosanac wrote:

Hi David,

it's all configured in your "redeliveryPolicy", where you instructed the
broker to try resending the message indefinitely.

See http://activemq.apache.org/message-redelivery-and-dlq-handling.html for
more information how redelivery works and how you can use "Dead letter
queues"

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

On Sat, Nov 7, 2009 at 7:35 PM, David Erickson <derickso@stanford.edu>wrote:

>
> Hi All,
> I am using AMQ inside a Spring IoC container. It is a very simple setup
> with the following config:
>
> 
> <amq:broker useJmx="false" persistent="false">
> <amq:transportConnectors>
> <amq:transportConnector uri="tcp://localhost:0" />
> </amq:transportConnectors>
>
> </amq:broker>
>
> 
> <amq:queue id="mailDestination" physicalName="mailsender"/>
>
> 
> <amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost">
> <amq:redeliveryPolicy>
> <amq:redeliveryPolicy backOffMultiplier="2"
> useExponentialBackOff="true" initialRedeliveryDelay="1000"
> maximumRedeliveries="-1"/>
> </amq:redeliveryPolicy>
> </amq:connectionFactory>
>
> <bean id="myJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
> <property name="connectionFactory">
> 
> <bean class="org.apache.activemq.pool.PooledConnectionFactory">
> <property name="connectionFactory">
> <ref local="jmsFactory"/>
> </property>
> </bean>
> </property>
> <property name="pubSubDomain">
> <value>false</value>
> </property>
> </bean>
>
> <bean id="mailContainer"
> class="org.springframework.jms.listener.DefaultMessageListenerContainer">
> <property name="connectionFactory" ref="jmsFactory"/>
> <property name="destination" ref="mailDestination"/>
> <property name="messageListener" ref="mailConsumer"/>
> <property name="sessionTransacted" value="true"/>
> </bean>
>
> Along with one corresponding producer and consumer.
>
> My question is this, I noticed that my consumer was having problems sending
> one of the emails in the queue, it was throwing an exception, being placed
> back in the queue, then the entire queue was waiting the backoff time for
> this message to attempt to be delivered again, over and over again,
> producing a head of line blocking problem (no messages can be delivered
> until the one with the error is delivered).
>
> Is there a solution for this? Can the message that failed be pushed to a
> re-delivery queue, or a troubled queue rather than the main queue? Or can
> the consumer walk through the queue looking for a message that is ready to
> be delivered rather than blocking on the front message that isn't set to be
> delivered for some long amount of time because of a previous failure and
> backoff time?
>
> Thanks!
> --
> View this message in context:
> http://old.nabble.com/Message-Head-of-line-blocking-tp26247283p26247283.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

-----
Dejan Bosanac

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

Message delivering.

2009-11-09T07:58:41Z

Hello everybody!
I'm new to activemq and I have a question. I have a publisher and subcriber to the topic, when I launch publisher it sends some messages to the topic, then I close that publisher and start subscriber and he doesn't get messages that publisher sended early. I have examples from here . Also I'm using persistence
<persistenceAdapter>
<amqPersistenceAdapter directory="somefolder"/>
</persistenceAdapter>

What classes should I use or ActiveMQ configuration? Help, please.
Thanks in advance.

Activemq-cpp and Message Groups malfunction

2009-11-09T07:02:02Z

Hello,

We are having a problem when using Activemq-cpp (version 2.2.6). This problem happens with Stomp and Openwire.
We create thousands of message groups dynamically : Create a message group, send several messages, close message group. The producer is a C++ application, the consumer is two instances of JBoss in two different machines. This "in general" works, but from time to time, in a random way, the message group that is assigned to one instance of JBoss is switched to the other instance, as if the previous one had failed.
Example :
* We send a message with some JMSXGroupId, it is sent to JBoss1
* We send several more messages, they are received by JBoss1, as expected.
* We keep sending messages, but now they are received by JBoss2!!??

JBoss1 is all the time up and running. The garbage collector didn't work during the test. This problem happens even with very low load, in a randomly way. Just sometimes. When we put a very high load, so the response time from JBoss increases, we start seeing many more of this "unexpected switching".
We checked the messages sent, to check the JMSXGroupId and JMSXGroupSeq (we even used a network sniffer to check them) and their values are right.

It looks as if ActiveMQ considers JBoss1 dead, so it switches the message groups to JBoss2. What is the criteria used by ActiveMQ to consider a consumer "dead" ? Is there some configuration parameter to increase any timeout waiting for acknowledge messages from consumers?

The strangest part is that if we use a Java application as the producer, it works perfect. There are never "unexpected switching". We compared the message properties sent in Java and C++ and they are the same.

We are so far completely lost at this.

Any idea?

Thanks,

Luis

Re: Out of Memory on 5.3

2009-11-09T07:01:51Z

fyi: you can disable periodic message expiry processing using a
destination policy entry that sets expireMessagesPeriod = 0

2009/11/9 afei <afei1689@...>:

>
>
> when a large of messages in queue,and no consumer or the consumer is very
> slow, the OOM problem occur, because :
> in org.apache.activemq.broker.region.Queue,the 588 line is :
> doBrowse(true, browsedMessages, this.getMaxExpirePageSize());
> ,transform to :
> doBrowse(false, browsedMessages, this.getMaxExpirePageSize());
> is ok.
>
>
> Dejan Bosanac wrote:
>>
>> Hi Mitch,
>>
>> yeah, I said in thread I was referring to, that it is working with
>> "regular"
>> stomp connector. I started investigating AMQ-2440 patch the other day,
>> should be have something soon.
>>
>> Cheers
>> --
>> Dejan Bosanac - http://twitter.com/dejanb
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
>>
>> On Wed, Oct 28, 2009 at 6:18 PM, Mitch Granger
>> <mitch.granger@...>wrote:
>>
>>> So we turned off stomp+nio and went back to plain old stomp and so far
>>> it's
>>> working fine. New(IO) isn't always better, I guess :-)
>>>
>>> Seems like maybe it's this issue ->
>>> https://issues.apache.org/activemq/browse/AMQ-2440
>>>
>>>
>>> afei wrote:
>>>
>>>> i have same problem
>>>>
>>>> http://www.nabble.com/file/p26093204/aaaaaa.jpg aaaaaa.jpg
>>>>
>>>>
>>>> themitchy wrote:
>>>>
>>>>> This is what we've done to tune so far:
>>>>>
>>>>> - UseDedicatedTaskRunner=false
>>>>> - flow control is off
>>>>> - stomp transport uses transport.closeAsync=false
>>>>>
>>>>> I agree that it is because of the high number of open/close connections
>>>>> from Stomp. When we monitor through JConsole we can see more threads
>>>>> starting up for each new connection. The problem is that these threads
>>>>> don't get let go. Even though the stomp clients are disconnecting the
>>>>> number of threads that get released is less than the number created.
>>>>> So
>>>>> the thread count goes up and up until it fails. All of the above
>>>>> settings/tuning only delay when it will hit the wall.
>>>>>
>>>>> Dejan Bosanac wrote:
>>>>>
>>>>>> Hi Mitch,
>>>>>>
>>>>>> I think the root cause of this problem is that you probably have Stomp
>>>>>> clients that open/close connection at a high rate. I simulated this
>>>>>> problem
>>>>>> on OSX with a StompLoadTest (
>>>>>>
>>>>>> http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/transport/stomp/StompLoadTest.java?view=log
>>>>>> ),
>>>>>> while trying to reproduce "too many open files" problem. You can find
>>>>>> some
>>>>>> of my findings (and workaround) in this thread.
>>>>>>
>>>>>>
>>>>>> http://www.nabble.com/%22too-many-open-files%22-error-with-5.3-and-Stomp-tt25888831.html#a26010080
>>>>>>
>>>>>> (BTW. it is producing "too many open files" problem on linux)
>>>>>> Basically,
>>>>>> the
>>>>>> problem with stomp is that every send is done in separate connection
>>>>>> and
>>>>>> thus considered to be a new producer for every message. So when
>>>>>> producer
>>>>>> flow control is hit, the producers are piling up and probably not
>>>>>> releasing
>>>>>> connections. Thus you can observe large number of tcp connections on
>>>>>> the
>>>>>> system in state TIME_WAIT (and TIME_CLOSE), which causes that the
>>>>>> system
>>>>>> limit is hit at one point. In the above thread, you can find a
>>>>>> workaround
>>>>>> that worked for me for that test. I started investigating this more
>>>>>> and
>>>>>> hopefully I'll have some more findings in the near future.
>>>>>>
>>>>>> Cheers
>>>>>> --
>>>>>> Dejan Bosanac - http://twitter.com/dejanb
>>>>>>
>>>>>> Open Source Integration - http://fusesource.com/
>>>>>> ActiveMQ in Action - http://www.manning.com/snyder/
>>>>>> Blog - http://www.nighttale.net
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 27, 2009 at 1:07 AM, Mitch Granger
>>>>>> <mitch.granger@...>wrote:
>>>>>>
>>>>>> Update: We've [nearly] proven that this only happens with AMQ running
>>>>>>> on
>>>>>>> openVZ. What exactly is causing it, we're still not sure. After
>>>>>>> memoryUsage is met, the number of threads skyrockets until we get
>>>>>>> OutOfMemoryError.
>>>>>>>
>>>>>>> It works just fine on regular hardware; We're going to try VMWare
>>>>>>> tomorrow.
>>>>>>>
>>>>>>> One thing really worth mentioning is that by using the fileCursor we
>>>>>>> actually started seeing it use the Temp Store. When reading about
>>>>>>> systemUsage it is NOT intuitive that the Temp Store does not come
>>>>>>> into
>>>>>>> play
>>>>>>> with the default cursor. Anyone keeping a significant volume of
>>>>>>> messages on
>>>>>>> their queues should be well served by changing the cursor.
>>>>>>>
>>>>>>>
>>>>>>> Mitch Granger wrote:
>>>>>>>
>>>>>>> Config is attached. We have also tried the activemq-scalability.xml
>>>>>>>> with
>>>>>>>> the only change being adding a stomp connector.
>>>>>>>>
>>>>>>>> Once we hit the memoryUsage limit we can [sometimes] connect new
>>>>>>>> consumers
>>>>>>>> but nothing comes back after we send the SUBSCRIBE frame.
>>>>>>>>
>>>>>>>> I expect sending to fail when we hit this limit but if we can't
>>>>>>>> subscribe
>>>>>>>> there's no chance of recovering from this state.
>>>>>>>>
>>>>>>>> Rob Davies wrote:
>>>>>>>>
>>>>>>>> On 26 Oct 2009, at 17:38, themitchy wrote:
>>>>>>>>>
>>>>>>>>> We're using only persistent messages and heap size is set to 2GB
>>>>>>>>> yet
>>>>>>>>> we
>>>>>>>>>
>>>>>>>>>> hit
>>>>>>>>>> the memoryUsage limit quite quickly (system usage config below).
>>>>>>>>>> This
>>>>>>>>>> is
>>>>>>>>>> followed by "java.lang.OutOfMemoryError: unable to create new
>>>>>>>>>> native
>>>>>>>>>> thread"
>>>>>>>>>> as the process quickly reaches the 2GB of heap we gave it. How
>>>>>>>>>> are
>>>>>>>>>> we
>>>>>>>>>> getting to that point with the memoryUsage limit set far below it?
>>>>>>>>>>
>>>>>>>>>> Is there no way to get AMQ to gracefully limit it's memory usage?
>>>>>>>>>>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <systemUsage>
>>>>>>>>>> <memoryUsage>
>>>>>>>>>> <memoryUsage limit="256 mb"/>
>>>>>>>>>> </memoryUsage>
>>>>>>>>>> <storeUsage>
>>>>>>>>>> <storeUsage limit="60 gb" name="foo"/>
>>>>>>>>>> </storeUsage>
>>>>>>>>>> <tempUsage>
>>>>>>>>>> <tempUsage limit="60 gb"/>
>>>>>>>>>> </tempUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>> </systemUsage>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Out-of-Memory-on-5.3-tp26064098p26064098.html
>>>>>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Can you send the rest of your config ?
>>>>>>>>>
>>>>>>>>> Rob Davies
>>>>>>>>> http://twitter.com/rajdavies
>>>>>>>>> I work here: http://fusesource.com
>>>>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>
>>
>> -----
>> Dejan Bosanac
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
> http://old.nabble.com/file/p26264779/dump.jpg
> --
> View this message in context: http://old.nabble.com/Out-of-Memory-on-5.3-tp26064098p26264779.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Re: Out of Memory on 5.3

2009-11-09T06:39:30Z

Looks like AMQ-2468 does not resolve this as there is also an issue
with the message list. I have opened
https://issues.apache.org/activemq/browse/AMQ-2481 to track this and
already have a reproducible test case.
Thanks for the heads up.

2009/11/9 afei <afei1689@...>:

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Error getting ActiveMQTopicSession (Multicast Transport)

2009-11-09T06:07:21Z

Hi,

We are using Multicast Transport to broadcasting messages. We get topicConnection successfully. But when it tries to get topicSession, it receives timeout. Do we need to pass any particular options to the connection or any other configuration might be wrong?

topicConnection = topicConnectionFactory.createTopicConnection(); //Successful
topicSession = topicConnection.createTopicSession(false, Session.AUTO_ACKNOWLEDGE);

Here is the stack trace.

2009/11/09 08:43:18 ERROR com.visioninfo.MessageManager - JMS Exception occurred
org.apache.activemq.ConnectionFailedException: The JMS connection has failed: Receive timed out
at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1320)
at org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:299)
at org.apache.activemq.ActiveMQConnection.createTopicSession(ActiveMQConnection.java:1047)
at com.visioninfo.MessageManager.<init>(MessageManager.java:147)
at com.visioninfo.MessageManager.initManager(MessageManager.java:92)
at com.visioninfo.web.WebApplicationListener.contextInitialized(WebApplicationListener.java:95)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3934)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4429)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:630)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:556)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:491)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:516)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.net.SocketTimeoutException: Receive timed out
at java.net.PlainDatagramSocketImpl.receive0(Native Method)
at java.net.PlainDatagramSocketImpl.receive(Unknown Source)
at java.net.DatagramSocket.receive(Unknown Source)
at org.apache.activemq.transport.udp.CommandDatagramSocket.read(CommandDatagramSocket.java:69)
at org.apache.activemq.transport.udp.UdpTransport.run(UdpTransport.java:147)

Thanks in advance!!

Re: Out of Memory on 5.3

2009-11-09T04:12:43Z

Think this is resolved by
https://issues.apache.org/activemq/browse/AMQ-2468 which is fixed on
trunk and the 5.3 branch. The pagedInPendingDispatch list which was
accumulating messages is now limited. This was highlighted by the jmx
purge operation but the same logic holds for the browse that is used
by there expiry task.
Can you validate with a current 5.4-SNAPSHOT?

2009/11/9 afei <afei1689@...>:

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Re: Out of Memory on 5.3

2009-11-09T03:24:34Z

when a large of messages in queue,and no consumer or the consumer is very slow, the OOM problem occur, because :
in org.apache.activemq.broker.region.Queue,the 588 line is :
doBrowse(true, browsedMessages, this.getMaxExpirePageSize());
,transform to :
doBrowse(false, browsedMessages, this.getMaxExpirePageSize());
is ok.

Dejan Bosanac wrote:

Hi Mitch,

yeah, I said in thread I was referring to, that it is working with "regular"
stomp connector. I started investigating AMQ-2440 patch the other day,
should be have something soon.

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

On Wed, Oct 28, 2009 at 6:18 PM, Mitch Granger <mitch.granger@sophos.com>wrote:

> So we turned off stomp+nio and went back to plain old stomp and so far it's
> working fine. New(IO) isn't always better, I guess :-)
>
> Seems like maybe it's this issue ->
> https://issues.apache.org/activemq/browse/AMQ-2440
>
>
> afei wrote:
>
>> i have same problem
>>
>> http://www.nabble.com/file/p26093204/aaaaaa.jpg aaaaaa.jpg
>>
>>
>> themitchy wrote:
>>
>>> This is what we've done to tune so far:
>>>
>>> - UseDedicatedTaskRunner=false
>>> - flow control is off
>>> - stomp transport uses transport.closeAsync=false
>>>
>>> I agree that it is because of the high number of open/close connections
>>> from Stomp. When we monitor through JConsole we can see more threads
>>> starting up for each new connection. The problem is that these threads
>>> don't get let go. Even though the stomp clients are disconnecting the
>>> number of threads that get released is less than the number created. So
>>> the thread count goes up and up until it fails. All of the above
>>> settings/tuning only delay when it will hit the wall.
>>>
>>> Dejan Bosanac wrote:
>>>
>>>> Hi Mitch,
>>>>
>>>> I think the root cause of this problem is that you probably have Stomp
>>>> clients that open/close connection at a high rate. I simulated this
>>>> problem
>>>> on OSX with a StompLoadTest (
>>>>
>>>> http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/transport/stomp/StompLoadTest.java?view=log
>>>> ),
>>>> while trying to reproduce "too many open files" problem. You can find
>>>> some
>>>> of my findings (and workaround) in this thread.
>>>>
>>>>
>>>> http://www.nabble.com/%22too-many-open-files%22-error-with-5.3-and-Stomp-tt25888831.html#a26010080
>>>>
>>>> (BTW. it is producing "too many open files" problem on linux) Basically,
>>>> the
>>>> problem with stomp is that every send is done in separate connection and
>>>> thus considered to be a new producer for every message. So when producer
>>>> flow control is hit, the producers are piling up and probably not
>>>> releasing
>>>> connections. Thus you can observe large number of tcp connections on the
>>>> system in state TIME_WAIT (and TIME_CLOSE), which causes that the system
>>>> limit is hit at one point. In the above thread, you can find a
>>>> workaround
>>>> that worked for me for that test. I started investigating this more and
>>>> hopefully I'll have some more findings in the near future.
>>>>
>>>> Cheers
>>>> --
>>>> Dejan Bosanac - http://twitter.com/dejanb
>>>>
>>>> Open Source Integration - http://fusesource.com/
>>>> ActiveMQ in Action - http://www.manning.com/snyder/
>>>> Blog - http://www.nighttale.net
>>>>
>>>>
>>>> On Tue, Oct 27, 2009 at 1:07 AM, Mitch Granger
>>>> <mitch.granger@sophos.com>wrote:
>>>>
>>>> Update: We've [nearly] proven that this only happens with AMQ running
>>>>> on
>>>>> openVZ. What exactly is causing it, we're still not sure. After
>>>>> memoryUsage is met, the number of threads skyrockets until we get
>>>>> OutOfMemoryError.
>>>>>
>>>>> It works just fine on regular hardware; We're going to try VMWare
>>>>> tomorrow.
>>>>>
>>>>> One thing really worth mentioning is that by using the fileCursor we
>>>>> actually started seeing it use the Temp Store. When reading about
>>>>> systemUsage it is NOT intuitive that the Temp Store does not come into
>>>>> play
>>>>> with the default cursor. Anyone keeping a significant volume of
>>>>> messages on
>>>>> their queues should be well served by changing the cursor.
>>>>>
>>>>>
>>>>> Mitch Granger wrote:
>>>>>
>>>>> Config is attached. We have also tried the activemq-scalability.xml
>>>>>> with
>>>>>> the only change being adding a stomp connector.
>>>>>>
>>>>>> Once we hit the memoryUsage limit we can [sometimes] connect new
>>>>>> consumers
>>>>>> but nothing comes back after we send the SUBSCRIBE frame.
>>>>>>
>>>>>> I expect sending to fail when we hit this limit but if we can't
>>>>>> subscribe
>>>>>> there's no chance of recovering from this state.
>>>>>>
>>>>>> Rob Davies wrote:
>>>>>>
>>>>>> On 26 Oct 2009, at 17:38, themitchy wrote:
>>>>>>>
>>>>>>> We're using only persistent messages and heap size is set to 2GB yet
>>>>>>> we
>>>>>>>
>>>>>>>> hit
>>>>>>>> the memoryUsage limit quite quickly (system usage config below).
>>>>>>>> This
>>>>>>>> is
>>>>>>>> followed by "java.lang.OutOfMemoryError: unable to create new native
>>>>>>>> thread"
>>>>>>>> as the process quickly reaches the 2GB of heap we gave it. How are
>>>>>>>> we
>>>>>>>> getting to that point with the memoryUsage limit set far below it?
>>>>>>>>
>>>>>>>> Is there no way to get AMQ to gracefully limit it's memory usage?
>>>>>>>>
>>>>>>>> <systemUsage>
>>>>>>>> <systemUsage>
>>>>>>>> <memoryUsage>
>>>>>>>> <memoryUsage limit="256 mb"/>
>>>>>>>> </memoryUsage>
>>>>>>>> <storeUsage>
>>>>>>>> <storeUsage limit="60 gb" name="foo"/>
>>>>>>>> </storeUsage>
>>>>>>>> <tempUsage>
>>>>>>>> <tempUsage limit="60 gb"/>
>>>>>>>> </tempUsage>
>>>>>>>> </systemUsage>
>>>>>>>> </systemUsage>
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://www.nabble.com/Out-of-Memory-on-5.3-tp26064098p26064098.html
>>>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>>> Can you send the rest of your config ?
>>>>>>>
>>>>>>> Rob Davies
>>>>>>> http://twitter.com/rajdavies
>>>>>>> I work here: http://fusesource.com
>>>>>>> My Blog: http://rajdavies.blogspot.com/
>>>>>>> I'm writing this: http://www.manning.com/snyder/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>
>>

-----
Dejan Bosanac

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

Re: Network broker and filter predicate

2009-11-09T02:31:39Z

Just a though, with the latest 5.3.0 release, the wireformat version
is 5. Enable debug logging to validate the wireformat negotiation
between the brokers to see what is the problem. Possibly there is some
mix up in the classpath or something.

2009/11/9 hackingbear <hackingbear@...>:

>
> Hi,
>
> In our setup, we have a backbone broker running the verbatim AMQ 5.3 and a
> 5.3 broker without our own BrokerFilter (gateway broker.) The gateway broker
> is for serving clients connecting from the Internet. One thing the broker
> does is to ensure only the right messages can be consumed, like:
>
> public Subscription addConsumer(ConnectionContext context, ConsumerInfo
> info) throws Exception {
> PropertyExpression filterProp = new
> PropertyExpression("FILTER_VALUE");
> List<String> elements = Arrays.asList("A", "B", "C", "D");
> BooleanExpression predicate =
> ComparisonExpression.createInFilter(filterProp, elements);
> info.setAdditionalPredicate(predicate);
>
> The config file is currently very simple:
>
> <beans ...>
> <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"
> />
>
> <broker xmlns="http://activemq.apache.org/schema/core" brokerName="tsgw"
> plugins="#gatewayPlugin">
>
> <networkConnectors>
> <networkConnector uri="static:(tcp://localhost:61616)"
> duplex="true"/>
> </networkConnectors>
> <persistenceAdapter>
> <memoryPersistenceAdapter />
> </persistenceAdapter>
>
> <transportConnectors>
> <transportConnector uri="tcp://localhost:62002" />
> </transportConnectors>
>
> </broker>
> <bean id="gatewayPlugin" class="amq.GatewayPlugin" />
> </beans>
>
> Then when the filter code runs, I will get an exception shown below, which
> also seems to throw the gateway broker in disarray as it keeps trying the
> same thing with same errors. If I comment out the setaddtionalPredicate(),
> the error goes away.
>
> Looking around the AMQ codes, the boolean/unary expressions indeed do not
> implement the DataStructure and this marshaller indeed is trying to cast to
> DataStructure. Further, SimpleAuthorizationBroker just instantiates a
> BooleanExpression.
>
> So I wonder if this has to do with network broker setup. How can I fix/work
> around this problem?
>
> Greatly appreciate for any help!!
>
> ----
>
> WARN DemandForwardingBridge Caught an exception processing local command
> [@BrokerService] 2009-11-08 15:22:40,877
> java.lang.ClassCastException: org.apache.activemq.filter.UnaryExpression$2
> cannot be cast to org.apache.activemq.command.DataStructure
> at
> org.apache.activemq.openwire.v3.ConsumerInfoMarshaller.tightMarshal1(ConsumerInfoMarshaller.java:127)
> at
> org.apache.activemq.openwire.OpenWireFormat.tightMarshalNestedObject1(OpenWireFormat.java:397)
> at
> org.apache.activemq.openwire.v3.BaseDataStreamMarshaller.tightMarshalNestedObject1(BaseDataStreamMarshaller.java:131)
> at
> org.apache.activemq.openwire.v3.MessageMarshaller.tightMarshal1(MessageMarshaller.java:132)
> at
> org.apache.activemq.openwire.v3.ActiveMQMessageMarshaller.tightMarshal1(ActiveMQMessageMarshaller.java:77)
> at
> org.apache.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:228)
> at
> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:166)
> at
> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:233)
> at
> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:83)
> at
> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:100)
> at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
> at
> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
> at
> org.apache.activemq.network.DemandForwardingBridgeSupport.serviceLocalCommand(DemandForwardingBridgeSupport.java:647)
> at
> org.apache.activemq.network.DemandForwardingBridgeSupport$1.onCommand(DemandForwardingBridgeSupport.java:147)
> at
> org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:104)
> at
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at
> org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:113)
> at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
> at
> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
> at
> org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1207)
> at
> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:793)
> at
> org.apache.activemq.broker.TransportConnection.iterate(TransportConnection.java:830)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
> --
> View this message in context: http://old.nabble.com/Network-broker-and-filter-predicate-tp26259578p26259578.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Re: Message Head of line blocking

2009-11-09T01:18:34Z

Hi David,

it's all configured in your "redeliveryPolicy", where you instructed the
broker to try resending the message indefinitely.

See http://activemq.apache.org/message-redelivery-and-dlq-handling.html for
more information how redelivery works and how you can use "Dead letter
queues"

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

On Sat, Nov 7, 2009 at 7:35 PM, David Erickson <derickso@...>wrote:

>
> Hi All,
> I am using AMQ inside a Spring IoC container. It is a very simple setup
> with the following config:
>
> 
> <amq:broker useJmx="false" persistent="false">
> <amq:transportConnectors>
> <amq:transportConnector uri="tcp://localhost:0" />
> </amq:transportConnectors>
>
> </amq:broker>
>
> 
> <amq:queue id="mailDestination" physicalName="mailsender"/>
>
> 
> <amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost">
> <amq:redeliveryPolicy>
> <amq:redeliveryPolicy backOffMultiplier="2"
> useExponentialBackOff="true" initialRedeliveryDelay="1000"
> maximumRedeliveries="-1"/>
> </amq:redeliveryPolicy>
> </amq:connectionFactory>
>
> <bean id="myJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
> <property name="connectionFactory">
> 
> <bean class="org.apache.activemq.pool.PooledConnectionFactory">
> <property name="connectionFactory">
> <ref local="jmsFactory"/>
> </property>
> </bean>
> </property>
> <property name="pubSubDomain">
> <value>false</value>
> </property>
> </bean>
>
> <bean id="mailContainer"
> class="org.springframework.jms.listener.DefaultMessageListenerContainer">
> <property name="connectionFactory" ref="jmsFactory"/>
> <property name="destination" ref="mailDestination"/>
> <property name="messageListener" ref="mailConsumer"/>
> <property name="sessionTransacted" value="true"/>
> </bean>
>
> Along with one corresponding producer and consumer.
>
> My question is this, I noticed that my consumer was having problems sending
> one of the emails in the queue, it was throwing an exception, being placed
> back in the queue, then the entire queue was waiting the backoff time for
> this message to attempt to be delivered again, over and over again,
> producing a head of line blocking problem (no messages can be delivered
> until the one with the error is delivered).
>
> Is there a solution for this? Can the message that failed be pushed to a
> re-delivery queue, or a troubled queue rather than the main queue? Or can
> the consumer walk through the queue looking for a message that is ready to
> be delivered rather than blocking on the front message that isn't set to be
> delivered for some long amount of time because of a previous failure and
> backoff time?
>
> Thanks!
> --
> View this message in context:
> http://old.nabble.com/Message-Head-of-line-blocking-tp26247283p26247283.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

Dejan Bosanac

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

Re: getPropertyNames on ActiveMQMessage supported?

2009-11-09T01:12:06Z

You can try to copy() the message before resending it and see if that helps.

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

On Sat, Nov 7, 2009 at 5:48 AM, Robert Nicholson <robert.nicholson@...
> wrote:

> So I'm doing the following in some JMS code
>
> Enumeration headers = message.getPropertyNames();
> if (!headers.hasMoreElements()) {
> return;
> }
>
> However an Enumeration that is empty. Is that expected?
>
> I'm replaying messages from a Hermes Message store into an ActiveMQ Queue
> but it doesn't look like they are getting any properties set.
>
> timestamp on the message object does have a valid timestamp.
>
> I'm using 5.3-SNAPSHOT but
>
> I see this message earlier
>
> http://osdir.com/ml/java.geronimo.user/2006-03/msg00051.html
>

Dejan Bosanac

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net

ActiveMQ-CPP client connecting to an embedded ActiveMQ in JBoss

2009-11-08T23:58:27Z

Hi All,

I have followed the tutorial found at Integrating ActiveMQ with JBoss. I was able to successfully test the integration with the examples included with the ActiveMQ distribution.

I am using ActiveMQ-CPP in order to use my C++ codes as JMS clients and I used ActiveMQ as the message broker. I was able to successfully create and connect a Java code as a producer and a C++ code as a consumer and vice versa. In this situation, ActiveMQ was started as stand-alone application by using the script found in the bin folder of the ActiveMQ distribution.

However, when I tried to use and start JBoss with ActiveMQ embedded on it, my C++ clients was not able to connect to the ActiveMQ (or JBoss ?). There are no messages being produced or consumed. The C++ codes I used are those in the example folder included with ActiveMQ-CPP distribution.

JBoss is installed on a Windows machine and my C++ codes are on a Unix machine which are connected via LAN.

Any idea why messages are successfully produced and consumed when using ActiveMQ as standalone but fails when using JBoss with embedded ActiveMQ?

- Pat

Re: ObjectMessage in C/C++

2009-11-08T18:27:54Z

I see..
Thanks a lot Tim!

Timothy Bish wrote:

On Thu, 2009-11-05 at 22:57 -0800, thinkbox wrote:
> I see, that makes sense, I get the point.
> Thanks Tim!
>
> - Pat
>
> X.S. Just a side comment. Having said that ActiveMQ C++ cannot support
> ObjectMessage -- can we assume that ObjectMessage is reserved for future
> use? Or are we looking at the CMS version of ObjectMessage the same is it is
> implemented now?
>

The ObjectMessage type in CMS is simply a placeholder which allows us to
return your client an ObjectMessage instnace when that type of message
is received instead of crashing or silently throwing out the message.
This allows you to at least access the the Message Properties and know
that it was received.

Regards
Tim.

>
>
> Timothy Bish wrote:
> >
> > On Wed, 2009-11-04 at 19:20 -0800, thinkbox wrote:
> >> Hi Nate, I am also in the same situation with Joshua and is currently
> >> considering ActiveMQ-CPP for my C++ clients to be able to talk to my Java
> >> components. I am currently using version 2.2.6 of ActiveMQ-CPP, are there
> >> any updates if ObjectMessages is now supported?
> >>
> >
> > ActiveMQ C++ cannot support ObjectMessage. The serialized version of a
> > Java object is not readable by the C++ code, and can't be converted into
> > a C++ type. You need to perform some sort of transformation on the
> > Object before sending to a C++ client, such as converting it to XML and
> > sending it as a TextMessage, then your C++ code can turn it back into a
> > C++ object of your own design or just parse the XML for the data you are
> > after. A search for C++ XML Binding on Google will give you some
> > insights into how to accomplish that.
> >
> >> Also, say my serialized object is implemented below,
> >>
> >> public class Subscriber implements Serializable {
> >> private String name;
> >> private int age;
> >>
> >> ...
> >>
> >> //setters and getters method follows
> >> }
> >>
> >>
> >> On my C++ code, say I have created an equivalent Subscriber class, can't
> >> I
> >> just cast the returned object of getObject() into Subscriber?
> >>
> >> Thanks,
> >> Pat
> >>
> >>
> >> nmittler wrote:
> >> >
> >> > ActiveMQ-CPP doesn't support ObjectMessages. If you're stuck sending
> >> > ObjectMessages from Java, probably your best bet would be to use a
> >> > Camel message translator in the broker
> >> > (http://activemq.apache.org/camel/message-translator.html
> >> > ).
> >> >
> >> > Regards,
> >> > Nate
> >> >
> >> > On Jul 6, 2008, at 6:44 PM, Joshua Smith wrote:
> >> >
> >> >> ActiveMQ Users-
> >> >>
> >> >> I have a project coming up where we'll be using ActiveMQ as our JMS
> >> >> server
> >> >> between a Java application and a C/C++ application. I'd like to
> >> >> exchange
> >> >> ObjectMessages. Is there anything special (e.g. endian) that I need
> >> >> to be
> >> >> aware of? Does the C/C++ application have to have objects defined
> >> >> that are
> >> >> the equivalent of my Java objects?
> >> >>
> >> >>
> >> >> Thanks,
> >> >> Joshua Smith
> >> >
> >> >
> >> >
> >>
> > --
> > Tim Bish
> > http://fusesource.com
> > http://timbish.blogspot.com/
> >
> >
> >
> >
> >
>

Network broker and filter predicate

2009-11-08T16:27:50Z

Hi,

In our setup, we have a backbone broker running the verbatim AMQ 5.3 and a 5.3 broker without our own BrokerFilter (gateway broker.) The gateway broker is for serving clients connecting from the Internet. One thing the broker does is to ensure only the right messages can be consumed, like:

public Subscription addConsumer(ConnectionContext context, ConsumerInfo info) throws Exception {
PropertyExpression filterProp = new PropertyExpression("FILTER_VALUE");
List<String> elements = Arrays.asList("A", "B", "C", "D");
BooleanExpression predicate = ComparisonExpression.createInFilter(filterProp, elements);
info.setAdditionalPredicate(predicate);

The config file is currently very simple:

<beans ...>
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" />

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="tsgw" plugins="#gatewayPlugin">

<networkConnectors>
<networkConnector uri="static:(tcp://localhost:61616)" duplex="true"/>
</networkConnectors>
<persistenceAdapter>
<memoryPersistenceAdapter />
</persistenceAdapter>

<transportConnectors>
<transportConnector uri="tcp://localhost:62002" />
</transportConnectors>

</broker>
<bean id="gatewayPlugin" class="amq.GatewayPlugin" />
</beans>

Then when the filter code runs, I will get an exception shown below, which also seems to throw the gateway broker in disarray as it keeps trying the same thing with same errors. If I comment out the setaddtionalPredicate(), the error goes away.

Looking around the AMQ codes, the boolean/unary expressions indeed do not implement the DataStructure and this marshaller indeed is trying to cast to DataStructure. Further, SimpleAuthorizationBroker just instantiates a BooleanExpression.

So I wonder if this has to do with network broker setup. How can I fix/work around this problem?

Greatly appreciate for any help!!

----

WARN DemandForwardingBridge Caught an exception processing local command [@BrokerService] 2009-11-08 15:22:40,877
java.lang.ClassCastException: org.apache.activemq.filter.UnaryExpression$2 cannot be cast to org.apache.activemq.command.DataStructure
at org.apache.activemq.openwire.v3.ConsumerInfoMarshaller.tightMarshal1(ConsumerInfoMarshaller.java:127)
at org.apache.activemq.openwire.OpenWireFormat.tightMarshalNestedObject1(OpenWireFormat.java:397)
at org.apache.activemq.openwire.v3.BaseDataStreamMarshaller.tightMarshalNestedObject1(BaseDataStreamMarshaller.java:131)
at org.apache.activemq.openwire.v3.MessageMarshaller.tightMarshal1(MessageMarshaller.java:132)
at org.apache.activemq.openwire.v3.ActiveMQMessageMarshaller.tightMarshal1(ActiveMQMessageMarshaller.java:77)
at org.apache.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:228)
at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:166)
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:233)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:83)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:100)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.network.DemandForwardingBridgeSupport.serviceLocalCommand(DemandForwardingBridgeSupport.java:647)
at org.apache.activemq.network.DemandForwardingBridgeSupport$1.onCommand(DemandForwardingBridgeSupport.java:147)
at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:104)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:113)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1207)
at org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:793)
at org.apache.activemq.broker.TransportConnection.iterate(TransportConnection.java:830)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Re: Race conditions

2009-11-08T11:19:58Z

On Sun, Nov 8, 2009 at 6:33 AM, Wais <hackwais@...> wrote:

>
> Hi,
>
> I've got one machine (x) publish to activemq where there are 2 subscribers
> (y, z). y is a databse that logs all the events as they happen within this
> system, the z is a machine which acts in response to messages published by
> x.
>
> So, I am worried by this scenario.
>
> x publishes to active mq, z reacts and also publishes to activemq. This
> happens very quickly (probably < 1/5th of a millisecond). What I'm afraid
> off is that due to thread switching, the message by z may be picked up by y
> before the message by x. Does activemq guarantee the order of messages in
> this way?

If you need message ordering you can enable it using the
strictOrderDispatchPolicy for a given destination or set of
destinations:

http://activemq.apache.org/total-ordering.html

Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'

ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder

Race conditions

2009-11-08T06:33:17Z

Hi,

I've got one machine (x) publish to activemq where there are 2 subscribers (y, z). y is a databse that logs all the events as they happen within this system, the z is a machine which acts in response to messages published by x.

So, I am worried by this scenario.

x publishes to active mq, z reacts and also publishes to activemq. This happens very quickly (probably < 1/5th of a millisecond). What I'm afraid off is that due to thread switching, the message by z may be picked up by y before the message by x. Does activemq guarantee the order of messages in this way?

Thanks,

Wais

Message Head of line blocking

2009-11-07T10:35:53Z

Hi All,
I am using AMQ inside a Spring IoC container. It is a very simple setup with the following config:


<amq:broker useJmx="false" persistent="false">
<amq:transportConnectors>
<amq:transportConnector uri="tcp://localhost:0" />
</amq:transportConnectors>

</amq:broker>


<amq:queue id="mailDestination" physicalName="mailsender"/>


<amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost">
<amq:redeliveryPolicy>
<amq:redeliveryPolicy backOffMultiplier="2" useExponentialBackOff="true" initialRedeliveryDelay="1000" maximumRedeliveries="-1"/>
</amq:redeliveryPolicy>
</amq:connectionFactory>

<bean id="myJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory">

<bean class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="connectionFactory">
<ref local="jmsFactory"/>
</property>
</bean>
</property>
<property name="pubSubDomain">
<value>false</value>
</property>
</bean>

<bean id="mailContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsFactory"/>
<property name="destination" ref="mailDestination"/>
<property name="messageListener" ref="mailConsumer"/>
<property name="sessionTransacted" value="true"/>
</bean>

Along with one corresponding producer and consumer.

My question is this, I noticed that my consumer was having problems sending one of the emails in the queue, it was throwing an exception, being placed back in the queue, then the entire queue was waiting the backoff time for this message to attempt to be delivered again, over and over again, producing a head of line blocking problem (no messages can be delivered until the one with the error is delivered).

Is there a solution for this? Can the message that failed be pushed to a re-delivery queue, or a troubled queue rather than the main queue? Or can the consumer walk through the queue looking for a message that is ready to be delivered rather than blocking on the front message that isn't set to be delivered for some long amount of time because of a previous failure and backoff time?

Thanks!

getPropertyNames on ActiveMQMessage supported?

2009-11-06T20:48:16Z

So I'm doing the following in some JMS code

Enumeration headers = message.getPropertyNames();
if (!headers.hasMoreElements()) {
return;
}

However an Enumeration that is empty. Is that expected?

I'm replaying messages from a Hermes Message store into an ActiveMQ
Queue but it doesn't look like they are getting any properties set.

timestamp on the message object does have a valid timestamp.

I'm using 5.3-SNAPSHOT but

I see this message earlier

http://osdir.com/ml/java.geronimo.user/2006-03/msg00051.html

Re: How to avoid ActiveMQ rar and webapp jar collision

2009-11-06T15:39:14Z

Yes. I am on 5.3 and I have xbean spring jar.
I got around the classloader isolation by deploying rar as ear.

PG

Re: Receiving old messages on durable topics after application restart using ActiveMQ two node cluster

2009-11-06T10:34:14Z

I have noticed the same thing. If your failover connector includes the two nodes, then it is likely that your application - as a result of restarts - has pollinated both brokers with the same durable subscription. However, within the context of the network of brokers, they are viewed as different durable subscriptions for the same topic? I tried setting 'dynamicOnly' to 'true', but that did not make a difference. You can cut down on the liklihood of pollinating both brokers by setting 'randomize' to 'false'. This would force your consumer to always connect to the first broker on the list. Another option is to use 'virtual topics' in combination with 'dynamicOnly' being set to 'true'.

http://activemq.apache.org/virtual-destinations.html

Joe
http://www.ttmsolutions.com

Stefan Verhoeff wrote:

Hello,

We are running a two node cluster of ActiveMQ 5.2. Our application uses durable persistent topics, it is both the producer and consumer of the messages. The application is configured using the failover:// protocol to work with the cluster nodes. The JMS client uses Spring for configuration, it's done as described here:
http://stefanlearninglog.blogspot.com/2009/07/persistent-jms-topics-using-activemq.html

It all runs fine. But when the application is restarted we start receiving old messages that have been consumed in the past. The age of the messages seems to correspond to the time of the last restart (for example: 20 days). We suspect the cluster nodes are replicating and storing the messages correctly but the node that the app is using is not marking the messages as 'consumed' on the other node. When on restart the other node is chosen it starts sending all messages that were consumed the last run on the other node. Is there anything we might have missed in configuration?

Kind regards,
Stefan

Re: Receiving old messages on durable topics after application restart using ActiveMQ two node cluster

2009-11-06T09:31:58Z

Can you share your configuration to show the cluster setup and
destination policies?
Off hand I can't think of any misconfiguration that would behave like this.
Can you reproduce with a short up time?

2009/11/6 Stefan Verhoeff <Stefan.Verhoeff@...>:

>
> Hello,
>
> We are running a two node cluster of ActiveMQ 5.2. Our application uses
> durable persistent topics, it is both the producer and consumer of the
> messages. The application is configured using the failover:// protocol to
> work with the cluster nodes. The JMS client uses Spring for configuration,
> it's done as described here:
> http://stefanlearninglog.blogspot.com/2009/07/persistent-jms-topics-using-activemq.html
>
> It all runs fine. But when the application is restarted we start receiving
> old messages that have been consumed in the past. The age of the messages
> seems to correspond to the time of the last restart (for example: 20 days).
> We suspect the cluster nodes are replicating and storing the messages
> correctly but the node that the app is using is not marking the messages as
> 'consumed' on the other node. When on restart the other node is chosen it
> starts sending all messages that were consumed the last run on the other
> node. Is there anything we might have missed in configuration?
>
> Kind regards,
> Stefan
> --
> View this message in context: http://old.nabble.com/Receiving-old-messages-on-durable-topics-after-application-restart-using-ActiveMQ-two-node-cluster-tp26230816p26230816.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com