putting a message back

[andrewk]

Hey folks,

Looking at the stomp protocol there seems to be a bit of a gap that I wish there was a better way to fill.

Let's say I am subscribed to a queue '/queue/a', and I am using client ack mode so my connection will not be flooded by messages - so I only get one message at a time, as I acknowledge one, I will be sent the next,

Alright, so I get a message, process it a bit, decide there is a problem; I can't finish processing this message, and want to put it back on the queue so a different client (assuming this is not the only one) can get the message and process it.  If I ack the message, it will be removed from the queue and will be considered delivered, so that's not what I want.  If I don't ack, then I won't receive any more messages, and this current message won't be sent on to anyone else as the broker waits to hear of I will ever ack it, so that's not great either.

A couple of work arounds are possible now as I see it

1) Send an identical message back into the subscribed queue, then ack the current message to remove it from the queue, using a transaction for both the send and ack so they can be aborted if they don't both work.  

This mostly works, but what happens if the re-sent message gets processed before an abort is sent?  

Also, the message lost it's priority in the queue, and is now in the back - not ideal, though perhaps fine for many situations.  

The message is also not exactly identical, it will have a new message id, so any tracking of messages being done will treat this as a new message rather than an attempt to continue processing the original message (though this could be mitigated by adding the original message id as an additional header).

One advantage is that a retry counter could be added to the message so that the message would not remain in the queue forever (i.e. if themessage I get has a retry over some threshold, throw the mesage to some dead letter queue or notify a system admin in some way).


2) Kill the connection.  This will cause the message to be considered undelivered, instead of just waiting for ack, and so it can/will be sent out to a different subscribed client connection.

This works pretty well in that the original message gets resent rather than a duplicate with a different id and queue position.

The obvious disadvantage is that now the client, to do further processing, has to restablish a connection, and resubscribe to destinations, which takes several messages, and a bit of time.  It seems to me it would be better to get the same effect, but with a single message, and without having to close and reopen the connection.

Is there a better way, or is there a different way of thinking about this where this isn't really a problem?

-Andrew

[James.Strachan]

On 8/28/07, andrewk <andrew@...> wrote:

>
> Hey folks,
>
> Looking at the stomp protocol there seems to be a bit of a gap that I wish
> there was a better way to fill.
>
> Let's say I am subscribed to a queue '/queue/a', and I am using client ack
> mode so my connection will not be flooded by messages - so I only get one
> message at a time, as I acknowledge one, I will be sent the next,
>
> Alright, so I get a message, process it a bit, decide there is a problem; I
> can't finish processing this message, and want to put it back on the queue
> so a different client (assuming this is not the only one) can get the
> message and process it.  If I ack the message, it will be removed from the
> queue and will be considered delivered, so that's not what I want.  If I
> don't ack, then I won't receive any more messages, and this current message
> won't be sent on to anyone else as the broker waits to hear of I will ever
> ack it, so that's not great either.

FWIW the common MOM pattern for dealing with this is to have a 'dead
letter queue', basically another queue to put messages which could not
be processed. If you use the same queue, you could have an infinite
loop (the same message getting processed repeatedly by the same
consumer, causing thrashing).

If you really want to re-try the message again later, preserving the
order of the queue, then try using a transaction and do a rollback.

> A couple of work arounds are possible now as I see it
>
> 1) Send an identical message back into the subscribed queue, then ack the
> current message to remove it from the queue, using a transaction for both
> the send and ack so they can be aborted if they don't both work.
>
> This mostly works, but what happens if the re-sent message gets processed
> before an abort is sent?
>
> Also, the message lost it's priority in the queue, and is now in the back -
> not ideal, though perhaps fine for many situations.
>
> The message is also not exactly identical, it will have a new message id, so
> any tracking of messages being done will treat this as a new message rather
> than an attempt to continue processing the original message (though this
> could be mitigated by adding the original message id as an additional
> header).

FWIW if you want to correlate / track messages, you should add a
correlation identifier to the message (JMSCorrelationID)


--
James
-------
http://macstrac.blogspot.com/

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email