SwiftMQ 7.4.1 + spring 2.5 + transactions + ReplyTo == trouble
I seem to have run into a few bugs while getting a spring 2.5.x application working with SwiftMQ. The portion I've had a lot of troubles with listens to a queue and responds to the received message's ReplyTo destination.
I think there are three issues I've run into.
1. An object leak in SwiftMQ QueueReceiver objects when transactions are used and consumers (and sessions and connections) are not cached.
2. A bug of springsupport's SingleSharedConnectionFactory that causes all producers to be cached -- even if they're only used once to reply on a temporary queue named in a ReplyTo header. Since these producers never actually get closed, the max-producers limit for the connection is quickly reached.
3. A bug causing "transaction closed" exceptions after sending a response only when run on OSX. This occurs with both Java 1.5 and 1.6. This bug doesn't seem to happen on Linux.
I don't believe that these are spring bugs in general because these problems don't come up when using the same application on Active MQ -- not that Active MQ behavior is correct -- I've found too many bugs in that to make me want to use it for anything other than testing.
I've attached jar files containing source and binaries for a simple spring app to reproduce these issues.
To summarize, the behaviors I'm seeing are ...
No caching:
Leak QueueReceiver objects and eventually OOM
Cache connection only:
Leak QueueReceiver objects and eventually OOM
Cache connection and session only:
Leak QueueReceiver objects and eventually OOM
Cache connection, session and consumer:
working on linux, fails with transaction closed exception on osx
Use springsupport's SingleSharedConnectionFactory:
no way to disable producer caching which causes 1-time-use producers on temp ReplyTo queues to exceed connection limit.
The number of QueueReceiver objects continues to grow until OOM. This occurs with just one application connected to SwiftMQ and doesn't require the application to actually be receiving any messages -- it just has to poll for messages.
The leak can be easily detected using Java 1.6 using jmap -histo:live ...
agewi-app-qu02:router$ /usr/java/default/bin/jmap -histo:live <pid> | grep QueueReceiver
197: 8 640 com.swiftmq.swiftlet.queue.QueueReceiver
226: 8 384 com.swiftmq.swiftlet.queue.QueueReceiver$PullTransactionRecycler
agewi-app-qu02:router$ /usr/java/default/bin/jmap -histo:live <pid> | grep QueueReceiver
46: 173 13840 com.swiftmq.swiftlet.queue.QueueReceiver
65: 173 8304 com.swiftmq.swiftlet.queue.QueueReceiver$PullTransactionRecycler
agewi-app-qu02:router$ /usr/java/default/bin/jmap -histo:live <pid> | grep QueueReceiver
40: 251 20080 com.swiftmq.swiftlet.queue.QueueReceiver
52: 251 12048 com.swiftmq.swiftlet.queue.QueueReceiver$PullTransactionRecycler
agewi-app-qu02:router$ /usr/java/default/bin/jmap -histo:live <pid> | grep QueueReceiver
33: 359 28720 com.swiftmq.swiftlet.queue.QueueReceiver
45: 359 17232 com.swiftmq.swiftlet.queue.QueueReceiver$PullTransactionRecycler
agewi-app-qu02:router$ /usr/java/default/bin/jmap -histo:live <pid> | grep QueueReceiver
33: 398 31840 com.swiftmq.swiftlet.queue.QueueReceiver
43: 398 19104 com.swiftmq.swiftlet.queue.QueueReceiver$PullTransactionRecycler
And finally, here's the exception that occurs when consumer caching is enabled with transactions on OSX ...
2009-04-13 11:41:34,166 [dummyInquiryJmsContainer-1] INFO org.springframework.jms.listener.DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination 'game_item_insertable_iweb' - trying to recover.
Cause: Could not commit JMS transaction; nested exception is javax.jms.JMSException: com.swiftmq.swiftlet.queue.QueueTransactionClosedException: transaction is closed!
org.springframework.transaction.TransactionSystemException: Could not commit JMS transaction; nested exception is javax.jms.JMSException: com.swiftmq.swiftlet.queue.QueueTransactionClosedException: transaction is closed!
at org.springframework.jms.connection.JmsTransactionManager.doCommit(JmsTransactionManager.java:242)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:982)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:974)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:876)
at java.lang.Thread.run(Thread.java:613)
Caused by: javax.jms.JMSException: com.swiftmq.swiftlet.queue.QueueTransactionClosedException: transaction is closed!
at com.swiftmq.jms.ExceptionConverter.convert(Unknown Source)
at com.swiftmq.jms.v630.SessionImpl.commit(Unknown Source)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.jms.connection.CachingConnectionFactory$CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:332)
at $Proxy1.commit(Unknown Source)
at org.springframework.jms.connection.JmsTransactionManager.doCommit(JmsTransactionManager.java:236)
... 7 more
Thanks for your help,
Bryan