Queue does not drain completely.

[RakeshRay]

Just installed Active MQ 5.2.
Using Stomp, inserted 5000 msgs and try to read the same with
while ( my $msg = $stomp->read() ) {
        $stomp->commit();
        print $msg.$/;
}
It leaves 2 messages in the queue, just reads 4998 and comes out.

Any suggestions?

[mjustin]

I don't know the PHP Stomp client very well (but a Delphi implementation).

$stomp->read() seems to return immediately without waiting for the next message.
There might be a different method $stomp->read(receivetimeout), which waits for the given time (in milliseconds) before it gives up.

Michael

RakeshRay wrote:

Just installed Active MQ 5.2.
Using Stomp, inserted 5000 msgs and try to read the same with
while ( my $msg = $stomp->read() ) {
        $stomp->commit();
        print $msg.$/;
}
It leaves 2 messages in the queue, just reads 4998 and comes out.

Any suggestions?

[Roger Hoover]

Maybe you meant to call  $stomp->receive_frame() in the while loop instead?

On Wed, Jul 1, 2009 at 1:15 PM, RakeshRay <Rakesh.Ray@...> wrote:

>
> Just installed Active MQ 5.2.
> Using Stomp, inserted 5000 msgs and try to read the same with
> while ( my $msg = $stomp->read() ) {
>        $stomp->commit();
>        print $msg.$/;
> }
> It leaves 2 messages in the queue, just reads 4998 and comes out.
>
> Any suggestions?
> --
> View this message in context:
> http://www.nabble.com/Queue-does-not-drain-completely.-tp24296039p24296039.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

[RakeshRay]

Hi, Hoover.
We have written a wrapper around and internally it is using receive_frame()...Here is the wrapper code.


sub read {
        my ($self) = @_;
        # after reading, we also need to acknoledge the read, but keep them both as part of the same transaction
       
        # still need to define a way of signalling errors vs nothing to read
       
        return unless $self->{stomp}->can_read({ timeout => '5' });
       
        my $frame = $self->{stomp}->receive_frame;

        return unless $frame;
       
        my $msg = $frame->body;
        my $msgid = $frame->headers->{'message-id'};

        # sending an ACK with the message id and a transaction makes the read part of the transaction
        $frame = Net::Stomp::Frame->new( {
                command => 'ACK',
                headers => { transaction => $self->_tran(), 'message-id' => $msgid },
        } );
    $self->{stomp}->send_frame($frame);

        return $msg;
}

Roger Hoover wrote:

Maybe you meant to call  $stomp->receive_frame() in the while loop instead?

On Wed, Jul 1, 2009 at 1:15 PM, RakeshRay <[e-mail]> wrote:

>
> Just installed Active MQ 5.2.
> Using Stomp, inserted 5000 msgs and try to read the same with
> while ( my $msg = $stomp->read() ) {
>        $stomp->commit();
>        print $msg.$/;
> }
> It leaves 2 messages in the queue, just reads 4998 and comes out.
>
> Any suggestions?
> --
> View this message in context:
> http://www.nabble.com/Queue-does-not-drain-completely.-tp24296039p24296039.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

[RakeshRay]

Hi,
We have written a wrapper around and internally it is using using timeout feature.
What version of Active MQ are you using?

sub read {
        my ($self) = @_;
        # after reading, we also need to acknoledge the read, but keep them both as part of the same transaction
       
        # still need to define a way of signalling errors vs nothing to read
       
        return unless $self->{stomp}->can_read({ timeout => '5' });
       
        my $frame = $self->{stomp}->receive_frame;

        return unless $frame;
       
        my $msg = $frame->body;
        my $msgid = $frame->headers->{'message-id'};

        # sending an ACK with the message id and a transaction makes the read part of the transaction
        $frame = Net::Stomp::Frame->new( {
                command => 'ACK',
                headers => { transaction => $self->_tran(), 'message-id' => $msgid },
        } );
    $self->{stomp}->send_frame($frame);

        return $msg;
}




I don't know the PHP Stomp client very well (but a Delphi implementation).

$stomp->read() seems to return immediately without waiting for the next message.
There might be a different method $stomp->read(receivetimeout), which waits for the given time (in milliseconds) before it gives up.

Michael

RakeshRay wrote:

Just installed Active MQ 5.2.
Using Stomp, inserted 5000 msgs and try to read the same with
while ( my $msg = $stomp->read() ) {
        $stomp->commit();
        print $msg.$/;
}
It leaves 2 messages in the queue, just reads 4998 and comes out.

Any suggestions?

[mjustin]

I am using 5.2 and 5.3-Snapshot. Have you tried to test it with a smaller number of messages, and without transactions?

RakeshRay wrote:

Hi,
We have written a wrapper around and internally it is using using timeout feature.
What version of Active MQ are you using?

[RakeshRay]

I tried with 1000 messsages or 800 byte/each, that is small enough.
I removed can_read before read_frame and seems behaving okay. Reading more to figure out if can_read is really required in my case or not!
Thanks for the other suggestions, though.

I am using 5.2 and 5.3-Snapshot. Have you tried to test it with a smaller number of messages, and without transactions?

RakeshRay wrote:

Hi,
We have written a wrapper around and internally it is using using timeout feature.
What version of Active MQ are you using?

[mjustin]

If it depends on message number / message size, maybe the broker needs more buffer space, you can try increasing the memoryLimit in the activemq.xml file. It helped with a similar problem where the producer stopped after a small number of messages until I increased the value to 50mb.

        <!-- Destination specific policies using destination names or wildcards -->
        <destinationPolicy>
            <policyMap>
                <policyEntries>
                    <policyEntry queue=">" memoryLimit="5mb"/>
                    <policyEntry topic=">" memoryLimit="5mb">
                    </policyEntry>
                </policyEntries>
            </policyMap>
        </destinationPolicy>


I tried with 1000 messsages or 800 byte/each, that is small enough.
I removed can_read before read_frame and seems behaving okay. Reading more to figure out if can_read is really required in my case or not!
Thanks for the other suggestions, though.

mjustin wrote:

I am using 5.2 and 5.3-Snapshot. Have you tried to test it with a smaller number of messages, and without transactions?

RakeshRay wrote:

Hi,
We have written a wrapper around and internally it is using using timeout feature.
What version of Active MQ are you using?

[RakeshRay]

Thanks for the suggestions, I tried, but does not help.
It could be the problem with the Stomp protocol!.
Anyone using Stomp to Connect to use Active MQ?


If it depends on message number / message size, maybe the broker needs more buffer space, you can try increasing the memoryLimit in the activemq.xml file. It helped with a similar problem where the producer stopped after a small number of messages until I increased the value to 50mb.

        <!-- Destination specific policies using destination names or wildcards -->
        <destinationPolicy>
            <policyMap>
                <policyEntries>
                    <policyEntry queue=">" memoryLimit="5mb"/>
                    <policyEntry topic=">" memoryLimit="5mb">
                    </policyEntry>
                </policyEntries>
            </policyMap>
        </destinationPolicy>


I tried with 1000 messsages or 800 byte/each, that is small enough.
I removed can_read before read_frame and seems behaving okay. Reading more to figure out if can_read is really required in my case or not!
Thanks for the other suggestions, though.

mjustin wrote:

I am using 5.2 and 5.3-Snapshot. Have you tried to test it with a smaller number of messages, and without transactions?

RakeshRay wrote:

Hi,
We have written a wrapper around and internally it is using using timeout feature.
What version of Active MQ are you using?

[mjustin]

RakeshRay wrote:

Thanks for the suggestions, I tried, but does not help.
It could be the problem with the Stomp protocol!.
Anyone using Stomp to Connect to use Active MQ?

You could use the Stomp client tools on my web site. The download includes a ProducerTool and a ConsumerTool which can be configured using command line parameters. It is tested with ActiveMQ 5.X.

http://www.mikejustin.com/download/Habari-demo.zip

[Aleksandar Ivanisevic-3]

RakeshRay <Rakesh.Ray@...> writes:

> Thanks for the suggestions, I tried, but does not help.
> It could be the problem with the Stomp protocol!.
> Anyone using Stomp to Connect to use Active MQ?

I'm using STOMP with perl Net::Stomp and it is working reasonably good
for simple use. 5.3.SNAPSHOT is buggy once you go to more advanced
features (i.e. transactions). see i.e. AMQ-2280

BTW, I always use can_read before the actual read.

> mjustin wrote:
>>
>> If it depends on message number / message size, maybe the broker needs
>> more buffer space, you can try increasing the memoryLimit in the
>> activemq.xml file. It helped with a similar problem where the producer
>> stopped after a small number of messages until I increased the value to
>> 50mb.
>>
>>         <!-- Destination specific policies using destination names or
>> wildcards -->
>>         <destinationPolicy>
>>             <policyMap>
>>                 <policyEntries>
>>                     <policyEntry queue=">" memoryLimit="5mb"/>
>>                     <policyEntry topic=">" memoryLimit="5mb">
>>                     </policyEntry>
>>                 </policyEntries>
>>             </policyMap>
>>         </destinationPolicy>
>>
>>
>> RakeshRay wrote:
>>>
>>> I tried with 1000 messsages or 800 byte/each, that is small enough.
>>> I removed can_read before read_frame and seems behaving okay. Reading
>>> more to figure out if can_read is really required in my case or not!
>>> Thanks for the other suggestions, though.
>>>
>>> mjustin wrote:
>>>>
>>>> I am using 5.2 and 5.3-Snapshot. Have you tried to test it with a
>>>> smaller number of messages, and without transactions?
>>>>
>>>>
>>>> RakeshRay wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>> We have written a wrapper around and internally it is using using
>>>>> timeout feature.
>>>>> What version of Active MQ are you using?
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Queue-does-not-drain-completely.-tp24296039p24427058.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>

--
Gle, svaka cast tebi i tvom poslu ali kaj nas moras svaki puta dojit sa
tvojom placom i poslom itd. Pa svaki post koji postas prije ili kasnije
pocinje aludirati na tvoje vrhunaravne izvore prihoda i materijalnih dobara.
Daj se sredi malo. Mislim da na ama bas nikoga ne zanima koliko ti zaradujes
i kako trosis svoje pare. Tomislav Kralj - hr.comp.programiranje.baze