Blocking throttle

[ljorquera]

Hi!
 I am trying to use the Throttle pattern to satisfy an SLA. The situation is that I must send messages to a provider but I can not send messages faster than a SLA defined rate. The process that sends messages to the provider reads from an activemq queue so route this queue to another, set a Throttle with the SLA rate between the original queue and the new one and make the process read from these new queue.
 The problem is that if the consumer process crash and is restarted, it founds many messages in the queue, as the throttle continues to move messages between the queues at the specified rate. The process then sends all these messages, violating the SLA.
 Is there a way of preventing the throttle of continuing putting messages in the queue unless they are being consumed?
 Thanks in advance for your help. I am quite new to Camel, so sorry if the question is too obvious.
 

[willem.jiang]

Hi,
Current camel throttler doesn't support to block the thread as you want.

If you want to implement this kind of feature yourself, here is my
suggestion.

First you can limit the message queue size for the Queue which you send
the message to. In this way if the message queue is full, camel jms
producer can know the consumer is stop working.
You can leverage the Camel onException to let camel now this exception.

Or you can implement some other heart bit mechanism to let camel route
know if the consumer is crashed.

I think you can implement a pause able task executor and set it into the
JMSConfiguration, and camel will set into a Spring
MessageListenerContainer for camel-jms consumer to use.

When you know the consumer processor is crashed, you can pause the task
executor as you want , and restart it when you need.

[1] http://camel.apache.org/exception-clause.html

Willem
ljorquera wrote:

> Hi!
>  I am trying to use the Throttle pattern to satisfy an SLA. The situation is
> that I must send messages to a provider but I can not send messages faster
> than a SLA defined rate. The process that sends messages to the provider
> reads from an activemq queue so route this queue to another, set a Throttle
> with the SLA rate between the original queue and the new one and make the
> process read from these new queue.
>  The problem is that if the consumer process crash and is restarted, it
> founds many messages in the queue, as the throttle continues to move
> messages between the queues at the specified rate. The process then sends
> all these messages, violating the SLA.
>  Is there a way of preventing the throttle of continuing putting messages in
> the queue unless they are being consumed?
>  Thanks in advance for your help. I am quite new to Camel, so sorry if the
> question is too obvious.
>  

[ljorquera]

Hi Willem,
 Thanks a lot for your response. As you suggested, I limited the size of the receiving Queue via the consumer.maximumPendingMessageLimit option and it worked perfectly! When the consumer restarts it only has a few messages and does not violate the SLA.
 I could not do this via the camel destination options because this is not implemented yet as discussed here, so I had to create the Queue via the java interface before creating the route.
 Thanks again!

willem.jiang wrote:

Hi,
Current camel throttler doesn't support to block the thread as you want.

If you want to implement this kind of feature yourself, here is my
suggestion.

First you can limit the message queue size for the Queue which you send
the message to. In this way if the message queue is full, camel jms
producer can know the consumer is stop working.
You can leverage the Camel onException to let camel now this exception.

Or you can implement some other heart bit mechanism to let camel route
know if the consumer is crashed.

I think you can implement a pause able task executor and set it into the
JMSConfiguration, and camel will set into a Spring
MessageListenerContainer for camel-jms consumer to use.

When you know the consumer processor is crashed, you can pause the task
executor as you want , and restart it when you need.

[1] http://camel.apache.org/exception-clause.html

Willem
ljorquera wrote:
> Hi!
>  I am trying to use the Throttle pattern to satisfy an SLA. The situation is
> that I must send messages to a provider but I can not send messages faster
> than a SLA defined rate. The process that sends messages to the provider
> reads from an activemq queue so route this queue to another, set a Throttle
> with the SLA rate between the original queue and the new one and make the
> process read from these new queue.
>  The problem is that if the consumer process crash and is restarted, it
> founds many messages in the queue, as the throttle continues to move
> messages between the queues at the specified rate. The process then sends
> all these messages, violating the SLA.
>  Is there a way of preventing the throttle of continuing putting messages in
> the queue unless they are being consumed?
>  Thanks in advance for your help. I am quite new to Camel, so sorry if the
> question is too obvious.
>  

[willem.jiang]

I'm glade you finally find a solution here.
Please feel free to fill a JIRA[1], so we will not forget to add camel
destination options into camel.

[1] https://issues.apache.org/activemq/browse/CAMEL

Willem

ljorquera wrote:

> Hi Willem,
>  Thanks a lot for your response. As you suggested, I limited the size of the
> receiving Queue via the consumer.maximumPendingMessageLimit option and it
> worked perfectly! When the consumer restarts it only has a few messages and
> does not violate the SLA.
>  I could not do this via the camel destination options because this is not
> implemented yet as discussed
> http://mail-archives.apache.org/mod_mbox/camel-users/200812.mbox/%3C20919263.post@...%3E
> here , so I had to create the Queue via the java interface before creating
> the route.
>  Thanks again!
>
>
> willem.jiang wrote:
>> Hi,
>> Current camel throttler doesn't support to block the thread as you want.
>>
>> If you want to implement this kind of feature yourself, here is my
>> suggestion.
>>
>> First you can limit the message queue size for the Queue which you send
>> the message to. In this way if the message queue is full, camel jms
>> producer can know the consumer is stop working.
>> You can leverage the Camel onException to let camel now this exception.
>>
>> Or you can implement some other heart bit mechanism to let camel route
>> know if the consumer is crashed.
>>
>> I think you can implement a pause able task executor and set it into the
>> JMSConfiguration, and camel will set into a Spring
>> MessageListenerContainer for camel-jms consumer to use.
>>
>> When you know the consumer processor is crashed, you can pause the task
>> executor as you want , and restart it when you need.
>>
>> [1] http://camel.apache.org/exception-clause.html
>>
>> Willem
>> ljorquera wrote:
>>> Hi!
>>>  I am trying to use the Throttle pattern to satisfy an SLA. The situation
>>> is
>>> that I must send messages to a provider but I can not send messages
>>> faster
>>> than a SLA defined rate. The process that sends messages to the provider
>>> reads from an activemq queue so route this queue to another, set a
>>> Throttle
>>> with the SLA rate between the original queue and the new one and make the
>>> process read from these new queue.
>>>  The problem is that if the consumer process crash and is restarted, it
>>> founds many messages in the queue, as the throttle continues to move
>>> messages between the queues at the specified rate. The process then sends
>>> all these messages, violating the SLA.
>>>  Is there a way of preventing the throttle of continuing putting messages
>>> in
>>> the queue unless they are being consumed?
>>>  Thanks in advance for your help. I am quite new to Camel, so sorry if
>>> the
>>> question is too obvious.
>>>  
>>
>>
>