Messages out of sequence (jboss and swiftmq)

View: New views
20 Messages — Rating Filter:   Alert me  
< Prev | 1 - 2 | Next >

Messages out of sequence (jboss and swiftmq)

by Andersb :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

This is the situation.

I'm using jboss 4.0.5 and swiftmq 6.2.1, running as one process each

I have a session bean that sends a number of messages on a jms queue in one transaction. Each message is sent using a new QueueConnection, QueueSession and QueueSender instance.

My problem is that when the transaction is committed in jboss and the messages are visible in the queue there is something strange happening. Say that I send 5 msgs in the transaction. Then this is the order in the queue.

2, 3, 4, 5, 1

The first message always ends up as the last message in the queue, otherwise the order is correct.

Any ideas?

Thanks.

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Do you send the messages via SwiftMQ's JCA resource adapter? Do you use XA?

Re: Messages out of sequence (jboss and swiftmq)

by Andersb :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

1. Yes (swiftmq.rar deployed in jboss)

2. Yes.

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ok. What I don't understand is that you a) send all messages within 1 tx but b) use a new connection/session etc for each message. If you do that, you use 5 managed connections for the tx while each enlist (join) a separate branch in the XA tx. The transaction manager (JBoss) then drives the 2PC. I don't know in which sequence the TM calls prepare and I don't know if you can rely on the sequence here.

Re: Messages out of sequence (jboss and swiftmq)

by Andersb :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

1. SessionBean1 gets a call with an array of objects as parameters
2. for each entry in the array it:
2.1 does some logic
2.2 depending on the outcome it might call SessionBean2
2.3 from here a call to the sendJMSMessage method might be done but not always (sessionbean1 doesn't know about this)
3. The sendJMSMessage method gets a new connection etc. (as said before) and does the actual sending to the queue.

I tested this with websphere appserver now with the same result. Using the JCA resource adapter this time also and the same code.


Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

It requires too much effort to reproduce it (this is a free forum). As a workaround you might pass the session or producer object as a parameter to the sendJMSMessage method to ensure everything is done from the same session.

Re: Messages out of sequence (jboss and swiftmq)

by michaelzilske :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

We have exactly the same phenomenon here (although we are using WebSphere, not JBoss).

Moreover, it is not only an issue of sequence: Messages 2-5 are not sent within the XA transaction, but immediately (they even arrive when the transaction is rolled back)! That they arrive out of order is only a symptom.

We use multiple short-lived Connections within one transaction for the same reason as the original poster.

Do you perhaps already have a solution for this?

Thanks a lot,
Michael

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

You are using Websphere and SwiftMQ via our JCA resource adapter? Did you configure it similar to this How To?

Re: Messages out of sequence (jboss and swiftmq)

by michaelzilske :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks for your reply - yes, this looks almost exactly like our setup. We are using swiftmq.rar though, not swiftmq-ivm.rar, and the router is on a different machine.

Does this mean that the one-connection-per-message way cannot be used? :-/

Re: Messages out of sequence (jboss and swiftmq)

by michaelzilske :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

By the way: Two Connections, each from a different ConnectionFactory, sending messages to two different Destinations, seem to behave normally (both are part of the XA-transaction). Only when we get a Connection, send a message, close it and then get a new Connection from the same ConnectionFactory, that one seems to be in a new transaction.

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

This example does actually what you are doing:

  public void onMessage(Message inMessage)
  {

    try
    {
      System.out.println("Message received: " + inMessage);
      InitialContext ctx = new InitialContext();
      ConnectionFactory cf = (ConnectionFactory) ctx.lookup("java:comp/env/CF");
      Connection connection = cf.createConnection();
      Destination dest = (Destination) ctx.lookup("java:comp/env/testqueue2");
      Session session = connection.createSession(true, 0);
      MessageProducer producer = session.createProducer(dest);
      TextMessage msg = session.createTextMessage("FWD: " + ((TextMessage) inMessage).getText());
      producer.send(msg);
      producer.close();
      session.close();
      connection.close();
    } catch (Exception e)
    {
      e.printStackTrace();
    }
  }

However, since it is an MDB, the number of MDB instances actually decides about the message sequence in the final destination. If the number of instances is 1, I would expect to have the same sequence in the final destination as they were send to the MDB destination.

On the other hand - it is up to the JTA transaction manager when it runs the 2PC on a transaction. It may do it FIFO or LIFO or in parallel. The JEE spec states that messages which should be in sequence should be send in a single transaction.

But what I don't understand is why some of your messages are not sent within a XA transaction at all. This seems to be strange and you should actually see errors in Websphere's log files.

Re: Messages out of sequence (jboss and swiftmq)

by michaelzilske :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Oh, thanks again, but there is not even an MDB involved in our trouble. The simplest way for us to reproduce our problem is this (from a stateless session bean). Calling sendMessages() sends messages 1-5, one per Connection. Then it throws a RuntimeException() to roll back the transaction.

Message 1 does not arrive (expected), messages 2-5 do arrive (not expected).

This seems to be very similar to what the original poster was experiencing, only he didn't roll back. Message 1 probably arrived last because it was the only one within the XA transaction, hence sent on commit, whereas the others were sent immediately.

I'm checking server logs again....

public void sendMessages() {
        int howMany = 5;
        for (int i=1; i<=howMany; i++) {
            sendMessageAndCloseConnection(i, connectionFactory, destination);
        }
        throw new RuntimeException();
}


private void sendMessageAndCloseConnection (int count, ConnectionFactory connectionFactory, Destination destination) {
        Connection connection = null;
        Session session = null;
        MessageProducer producer = null;
        try {
            connection = connectionFactory.createConnection();
            connection.start();
            session = connection.createSession(true, Session.AUTO_ACKNOWLEDGE);
            producer = session.createProducer(destination);
            TextMessage textMessage0 = session.createTextMessage("test");
            producer.send(textMessage0);
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            try {
                if (producer != null) {
                    producer.close();
                }
                if (session != null) {
                    session.close();
                }
                if (connection != null) {
                    connection.close();
                }
            } catch (JMSException e) {
                throw new RuntimeException(e);
            }
        }
    }

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Which SwiftMQ release do you use? Do you use SwiftMQ HA Router?

Did you configure the RA to allow one phase optimization as described here. Please set AllowOnePhase to false and let me know if that works.

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Btw, I don't think that throwing a RuntimeException is the proper way to force a rollback.

Re: Messages out of sequence (jboss and swiftmq)

by michaelzilske :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Our client is using an older version but the problem is reproducible on my machine with a newly downloaded swiftmw_7_5_4_jmsplus_eval. I switched AllowOnePass from "true" to "false" but the problem is still there.

Of course

getSessionContext().setRollbackOnly();

is the right way to force a rollback. Still, all messages except the first one immediately appear in the queue...

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

You should actually see in the Websphere logs whether all 5 sends are associated with the same transaction branch.

We will test it here too. It would speed up things if you could post a fully functional ear including sources which we can deploy on Websphere. Otherwise you'd need to wait until this issue will be tested during our regular maintenance iteration.

Re: Messages out of sequence (jboss and swiftmq)

by michaelzilske :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thank you: Here is a minimal test case. The EAR file contains a stateless session bean which can be called with WebSphere's Universal Test Client.

1.) sendMessages(int howMany) sends some messages.
2.) sendMessagesAndRollback(int howMany) sends some messages and rolls back the transaction.

Observable behavior (using SwiftMQ explorer):

1.) sends howMany messages
2.) sends howMany-1 messages

The EAR has resource references CF and testqueue which default to jms/CF and jms/testqueue

jmsxatest.ear

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks. Unfortunately the Universal Test Client is only availabke with the toolkit which is for customers only. So we can't test that.

Re: Messages out of sequence (jboss and swiftmq)

by michaelzilske :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

OK, hold on. I'm putting a Servlet in front of it.

Re: Messages out of sequence (jboss and swiftmq)

by IIT Software :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks. If you could provide it quickly, we could test it right away...
< Prev | 1 - 2 | Next >