XAResourceRecovery for ActiveMQ on JBoss

[Scott McNab]

Hello all.

I am trying to get XAResource recovery to work with JBoss JTA and ActiveMQ,
with no success so far. The core of the problem seems to be this:

I cannot get the ActiveMQ JCA resource adaptor to bind an
XAConnectionFactory instance to JNDI - no matter how I configure the
resource adaptor, it always binds an instance of ActiveMQConnectionFactory,
instead of ActiveMQXAConnectionFactory.

It appears that many other users have raised this issue also, yet no-one
seems to have provided a solution:

    http://www.nabble.com/XAConnectionFactory-td8445158.html
    http://www.nabble.com/JBoss-and-XAConnectionFactory-td16961291.html

http://www.nabble.com/Using-JNDI-to-lookup-an-XAConnectionFactory-td10348154.html
    http://www.nabble.com/JNDI-activeMQ-and-XA-td16961307.html

Perhaps if I explain why I think the XAConnectionFactory is required,
someone might be able to show me how to get this binding? Alternatively, it
would be great if someone could explain why it is not needed, or provide
some other solution to getting full recoverable XA support with ActiveMQ and
JBoss!


Here is the backgound on where I am at and my workings so far. I apologise
if this is a little long, but I want to be complete so other people can
follow (and correct me if necessary!):

1. In order to support periodic recovery of aborted XA transactions, JBoss
JTA (arjuna) requires a "recovery module" to be provided for each
recoverable XA resource (e.g. datasources, JMS providers, etc)

2. To do this, I have edited the
jboss/server/all/conf/jboss-jta-properties.xml file to include the following
addtional line for ActiveMQ (in addition to the JBoss Messaging and
datasource recovery modules, which I also need):

    <properties depends="arjuna" name="jta">
        <property name="com.arjuna.ats.jta.recovery.XAResourceRecovery.AMQ"

value="org.jboss.jms.server.recovery.MessagingXAResourceRecovery;java:/AMQProvider"/>
        <property
name="com.arjuna.ats.jta.recovery.XAResourceRecovery.JBMESSAGING1"

value="org.jboss.jms.server.recovery.MessagingXAResourceRecovery;java:/DefaultJMSProvider"/>
        <property name="com.arjuna.ats.jta.recovery.XAResourceRecovery.JDBC"

value="com.arjuna.ats.internal.jbossatx.jta.AppServerJDBCXARecovery;DefaultDS"/>
...
    </properties>

3. This requires a corresponding JMSProviderLoader instance bound to
java:/AMQProvider in JNDI. To do this, I have deployed the following mbean
in the jboss/server/all/deploy/activemq-jms-ds.xml file:

  <!-- The JMS provider loader for ActiveMQ -->
  <mbean code="org.jboss.jms.jndi.JMSProviderLoader"
     name="jboss.jms:service=JMSProviderLoader,name=AMQProvider">
    <attribute name="ProviderName">AMQProvider</attribute>
    <attribute name="ProviderAdapterClass">
      org.jboss.jms.jndi.JNDIProviderAdapter
    </attribute>
    <!-- The combined connection factory -->
    <attribute name="FactoryRef">activemq/XAConnectionFactory</attribute>
    <!-- The queue connection factory -->
    <attribute
name="QueueFactoryRef">activemq/XAConnectionFactory</attribute>
    <!-- The topic factory -->
    <attribute
name="TopicFactoryRef">activemq/XAConnectionFactory</attribute>
    <!-- Access JMS via HAJNDI -->
    <attribute name="Properties">
       java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
       java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
       java.naming.provider.url=${jboss.bind.address:localhost}:1100
       jnp.disableDiscovery=true
       jnp.partitionName=${jboss.partition.name:DefaultPartition}
       jnp.maxRetries=1
    </attribute>
  </mbean>

4. This file also contains the connection factory deployment descriptor,
which binds (or is supposed to!) the XAConnectionFactory instance to JNDI:

  <tx-connection-factory>
    <jndi-name>activemq/XAConnectionFactory</jndi-name>
    <use-java-context>false</use-java-context>
    <xa-transaction/>
    <track-connection-by-tx/>
    <rar-name>activemq-ra.rar</rar-name>

<connection-definition>javax.jms.XAConnectionFactory</connection-definition>
    <min-pool-size>1</min-pool-size>
    <max-pool-size>200</max-pool-size>
    <blocking-timeout-millis>30000</blocking-timeout-millis>
    <idle-timeout-minutes>3</idle-timeout-minutes>

<security-domain-and-application>JmsXARealm</security-domain-and-application>
  </tx-connection-factory>

5. In addition to the activemq-jms-ds.xml deployment descriptor, I also have
the ActiveMQ JCA resource adaptor deployed as
jboss/server/all/deploy/activemq-ra.rar in exploded form so I can edit the
files.

6. I have modified jboss/server/all/deploy/activemq-ra.rar/META-INF/ra.xml
to include the following additional lines in the <outbound-resourceadapter>
section:

            <connection-definition>

<managedconnectionfactory-class>org.apache.activemq.ra.ActiveMQManagedConnectionFactory</managedconnectionfactor
y-class>

<connectionfactory-interface>javax.jms.XAConnectionFactory</connectionfactory-interface>

<connectionfactory-impl-class>org.apache.activemq.ra.ActiveMQXAConnectionFactory</connectionfactory-impl-class>

<connection-interface>javax.jms.XAConnection</connection-interface>

<connection-impl-class>org.apache.activemq.ra.ManagedConnectionProxy</connection-impl-class>
            </connection-definition>

If my understanding is correct, this is all I should need to do to get an
org.apache.activemq.ra.ActiveMQXAConnectionFactory instance bound in JNDI as
activemq/XAConnectionFactory - yet for some reason I get a non-XA
org.apache.activemq.ra.ActiveMQConnectionFactory instance instead!

This is confirmed by the following exception, which occurs when JBoss JTA
attempts to perform its periodic recovery:

08-12 17:40:36.250 DEBUG [ats.txoj.logging.txojLoggerI18N] (Thread-6:)
[com.arjuna.ats.internal.txoj.recovery.TORecoveryModule_6] -
TORecoveryModule - second pass
08-12 17:40:36.250 DEBUG [ats.jta.logging.loggerI18N] (Thread-6:)
[com.arjuna.ats.internal.jta.recovery.info.secondpass] Local
XARecoveryModule - second pass
08-12 17:40:36.250 ERROR [jms.server.recovery.MessagingXAResourceWrapper]
(Thread-6:) ********************************Failed to connect to server
java.lang.ClassCastException: Object at 'activemq/XAConnectionFactory' in
context
{java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory,
java.naming.provider.url=192.168.
100.33:1100, jnp.disableDiscovery=true, jnp.partitionName=DefaultPartition,
jnp.maxRetries=1,
java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces:org.jboss.naming:org.jnp.i
nterfaces, hostKey=192.168.100.33:1100} is not an instance of
[class=javax.jms.XAConnectionFactory
classloader=org.jboss.mx.loading.UnifiedClassLoader3@17ba38f{ url=null
,addedOrder=2}
interfaces={}] object class is
[class=org.apache.activemq.ra.ActiveMQConnectionFactory
classloader=org.jboss.mx.loading.UnifiedClassLoader3@16181be{
url=file:/C:/projects/ccs/jboss-eap-
4.3.0.GA/server/all/deploy/activemq-ra.rar/ ,addedOrder=29}
interfaces={interface=javax.jms.ConnectionFactory
classloader=org.jboss.mx.loading.UnifiedClassLoader3@17ba38f{ url=null ,add
edOrder=2}, interface=javax.jms.QueueConnectionFactory
classloader=org.jboss.mx.loading.UnifiedClassLoader3@17ba38f{ url=null
,addedOrder=2}, interface=javax.jms.TopicConnectionFactory
classloader=org.jboss.mx.loading.UnifiedClassLoader3@17ba38f{ url=null
,addedOrder=2}, interface=javax.resource.Referenceable
classloader=org.jboss.mx.loading.UnifiedClassLoader3@17ba38
f{ url=null ,addedOrder=2}, interface=java.io.Serializable
classloader=null}]
        at org.jboss.util.naming.Util.checkObject(Util.java:329)
        at org.jboss.util.naming.Util.lookup(Util.java:216)
        at
org.jboss.jms.server.recovery.MessagingXAResourceWrapper.getConnectionFactory(MessagingXAResourceWrapper.java:392)
        at
org.jboss.jms.server.recovery.MessagingXAResourceWrapper.connect(MessagingXAResourceWrapper.java:338)
        at
org.jboss.jms.server.recovery.MessagingXAResourceWrapper.getDelegate(MessagingXAResourceWrapper.java:260)
        at
org.jboss.jms.server.recovery.MessagingXAResourceWrapper.recover(MessagingXAResourceWrapper.java:108)
        at
com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecovery(XARecoveryModule.java:746)
        at
com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.resourceInitiatedRecovery(XARecoveryModule.java:685)
        at
com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkSecondPass(XARecoveryModule.java:179)
        at
com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWork(PeriodicRecovery.java:237)
        at
com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:163)
08-12 17:40:36.250 WARN  [ats.jta.logging.loggerI18N] (Thread-6:)
[com.arjuna.ats.internal.jta.recovery.xarecovery1] Local
XARecoveryModule.xaRecovery  got XA exception javax.transactio
n.xa.XAException: Error trying to connect to provider java:/AMQProvider,
XAException.XAER_RMERR

Notice this is a ClassCastException - the factory instance bound to
activemq/XAConnectionFactory does not support the XAConnectionFactory
interface, despite my <tx-connection-factory> deployment descriptor
requesting it, and the <outbound-resourceadapter> defining one!

I can confirm this by using the JMX console "JNDIView" to get a view of the
JNDI tree. Here you can clearly see the wrong concrete class instance is
bound:

+- activemq (class: org.jnp.interfaces.NamingContext)
 | +- XAConnectionFactory (class:
org.apache.activemq.ra.ActiveMQConnectionFactory)


From what I can tell, this behaviour is incorrect and is actually a bug in
the ActiveMQ JCA resource adaptor - it does not seem to honour the
<connectionfactory-impl-class> setting in the ra.xml file. (this is
confirmed when I set the classname to a bogus value and no error was
raised!).

Alternatively, perhaps I am missing something? Is there some other way to
get ActiveMQ to participate in JBoss JTA XAResource recovery?

FYI, I have spent a couple of days on this problem now and am stumped! I can
confirm the exact same behaviour exists with both ActiveMQ 4.1.1 and 5.2.0.

Can anyone help me here?

Many thanks!
Scott

[Roland.Lichti]

Hi Scott,

Scott McNab schrieb:

We have the same problem. I just checked the source of the RA and found
some not so nice information.

1. ActiveMQ has XA support.
2. The ActiveMQ RA does encapsulate it from the API. Every layer again
works only with the non-XA-versions of session, connection, ...

Therefore to get the XAConnectionFactory bound to JNDI a complete
rewrite of the RA is needed. Some concepts may be
even incompatible with that idea (like using the inbound session for
everything else).

I'm willing to do try it - but only if there is a chance to get it back
into SVN (I don't need to be the committer, somebody may check my code
and even refuse it if it is not up to the standards). But if the idea of
a full XA compatible RA is not wanted by the core team, it is not worth
the time to code it ...

bye,
    Roland

--
Roland Thomas Lichti
IT Application Manager

Telefónica o2 Germany GmbH & Co. OHG
Hülshorstweg 30, 33415 Verl
t: [phone], f: [phone]
roland.lichti@...
http://www.telefonica.de

Bitte finden Sie hier die
handelsrechtlichen Pflichtangaben: http://www.telefonica.de/pflichtangaben.html

[rajdavies]

On 5 Nov 2009, at 05:58, Roland T. Lichti wrote:

> Hi Scott,
>
> Scott McNab schrieb:
>
> We have the same problem. I just checked the source of the RA and  
> found
> some not so nice information.
>
> 1. ActiveMQ has XA support.
> 2. The ActiveMQ RA does encapsulate it from the API. Every layer again
> works only with the non-XA-versions of session, connection, ...
>
> Therefore to get the XAConnectionFactory bound to JNDI a complete
> rewrite of the RA is needed. Some concepts may be
> even incompatible with that idea (like using the inbound session for
> everything else).
>
> I'm willing to do try it - but only if there is a chance to get it  
> back
> into SVN (I don't need to be the committer, somebody may check my code
> and even refuse it if it is not up to the standards). But if the  
> idea of
> a full XA compatible RA is not wanted by the core team, it is not  
> worth
> the time to code it ...
>
> bye,
>    Roland
>
> --
> Roland Thomas Lichti
> IT Application Manager
>
> Telefónica o2 Germany GmbH & Co. OHG
> Hülshorstweg 30, 33415 Verl
> t: [phone], f: [phone]
> roland.lichti@...
> http://www.telefonica.de
>
> Bitte finden Sie hier die
> handelsrechtlichen Pflichtangaben: http://www.telefonica.de/pflichtangaben.html
>

Hi Roland,

would love to get the contribution!

cheers,

Rob

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/

[jp.belanger]

rajdavies wrote:

On 5 Nov 2009, at 05:58, Roland T. Lichti wrote:

> I'm willing to do try it - but only if there is a chance to get it  
> back
> into SVN (I don't need to be the committer, somebody may check my code
> and even refuse it if it is not up to the standards). But if the  
> idea of
> a full XA compatible RA is not wanted by the core team, it is not  
> worth
> the time to code it ...
>
> bye,
>    Roland

Hi Roland,

would love to get the contribution!

cheers,

Rob

Anything new on this topic?

I'm having transaction recovery problem in my production environment and it's getting bad.

I've tried making the RA adapter XA compatible but most of the Managed and proxy object don't fit very well with the XA versions of Connection, XASession and such. And lack the knowledge of the inner working of ActiveMQ to do this in a reasonable time.

[Gary Tully]

Do you have some broken test case? It would be great to have some concrete
examples of what does not work atm.

2009/12/9 jp.belanger <jp.belanger@...>

>
>
> rajdavies wrote:
> >
> >
> > On 5 Nov 2009, at 05:58, Roland T. Lichti wrote:
> >
> >> I'm willing to do try it - but only if there is a chance to get it
> >> back
> >> into SVN (I don't need to be the committer, somebody may check my code
> >> and even refuse it if it is not up to the standards). But if the
> >> idea of
> >> a full XA compatible RA is not wanted by the core team, it is not
> >> worth
> >> the time to code it ...
> >>
> >> bye,
> >>    Roland
> >
> > Hi Roland,
> >
> > would love to get the contribution!
> >
> > cheers,
> >
> > Rob
> >
> >
>
> Anything new on this topic?
>
> I'm having transaction recovery problem in my production environment and
> it's getting bad.
>
> I've tried making the RA adapter XA compatible but most of the Managed and
> proxy object don't fit very well with the XA versions of Connection,
> XASession and such. And lack the knowledge of the inner working of ActiveMQ
> to do this in a reasonable time.
>
> --
> View this message in context:
> http://old.nabble.com/XAResourceRecovery-for-ActiveMQ-on-JBoss-tp24933498p26712047.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
http://blog.garytully.com

Open Source Integration
http://fusesource.com

[Andreas Gies]

Hi there,

I am working on some JBoss integration as well. I don't understand the comment on
"making the RA XA compatible". At the end of the day it depends on hon how you do the
transaction management and define the connection factories.

A non-tx-connection-factory defined in the jboss-ds does not support transactions, a
tx-connection without the xa-transaction element does only support local transactions,
a tx-connection with the xa-connection element support local and xa-transactions.

As far as I am reading the code correctly, the RA recognizes the transaction context correctly
and encapsulates local AND xa transactions in org.apacheactivemq.TransactionContext.

I *think* you only need to use the XA capable connection factory if you want to register the
CF manually and not have the RA do the transaction management. So, either you are using the
RA with the correct definition in the ds-definition file OR you are using a XA capable CF.

Could you elaborate a bit on your use case in case I am missing something, perhaps there is
a test case to try ?


Best regards
Andreas

On Dec 9, 2009, at 4:27 PM, jp.belanger wrote:

>
>
> rajdavies wrote:
>>
>>
>> On 5 Nov 2009, at 05:58, Roland T. Lichti wrote:
>>
>>> I'm willing to do try it - but only if there is a chance to get it  
>>> back
>>> into SVN (I don't need to be the committer, somebody may check my code
>>> and even refuse it if it is not up to the standards). But if the  
>>> idea of
>>> a full XA compatible RA is not wanted by the core team, it is not  
>>> worth
>>> the time to code it ...
>>>
>>> bye,
>>>   Roland
>>
>> Hi Roland,
>>
>> would love to get the contribution!
>>
>> cheers,
>>
>> Rob
>>
>>
>
> Anything new on this topic?
>
> I'm having transaction recovery problem in my production environment and
> it's getting bad.
>
> I've tried making the RA adapter XA compatible but most of the Managed and
> proxy object don't fit very well with the XA versions of Connection,
> XASession and such. And lack the knowledge of the inner working of ActiveMQ
> to do this in a reasonable time.
>
> --
> View this message in context: http://old.nabble.com/XAResourceRecovery-for-ActiveMQ-on-JBoss-tp24933498p26712047.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

[jp.belanger]

From what I can see from my use, is the fact that the XARessource not being Serializable, the Recovery engine of JBoss fails reacquire the context for Transaction recovery.

I tried using a custom Transaction recovery based on the JBossMQ one. (MessagingXAResourceWrapper and MessagingXAResourceRecovery)

But the RA doesn't return a XAConnectionFactory. So I kinda get stuck there.

Andreas Gies wrote:

Hi there,

I am working on some JBoss integration as well. I don't understand the comment on
"making the RA XA compatible". At the end of the day it depends on hon how you do the
transaction management and define the connection factories.

A non-tx-connection-factory defined in the jboss-ds does not support transactions, a
tx-connection without the xa-transaction element does only support local transactions,
a tx-connection with the xa-connection element support local and xa-transactions.

As far as I am reading the code correctly, the RA recognizes the transaction context correctly
and encapsulates local AND xa transactions in org.apacheactivemq.TransactionContext.

I *think* you only need to use the XA capable connection factory if you want to register the
CF manually and not have the RA do the transaction management. So, either you are using the
RA with the correct definition in the ds-definition file OR you are using a XA capable CF.

Could you elaborate a bit on your use case in case I am missing something, perhaps there is
a test case to try ?