Get Queue Size in Pika (AMQP Python)

Question

Simple question, but Google or the Pika open source code did not help. Is there a way to query the current queue size (item counter) in Pika?

Answer 1

I know that this question is a bit old, but here is an example of doing this with pika.

Regarding AMQP and RabbitMQ, if you have already declared the queue, you can re-declare the queue with the passive flag on and keeping all other queue parameters identical. The response to this declaration declare-ok will include the number of messages in the queue.

Here is an example With pika 0.9.5:

import pika

def on_callback(msg):
    print msg

params = pika.ConnectionParameters(
        host='localhost',
        port=5672,
        credentials=pika.credentials.PlainCredentials('guest', 'guest'),
    )

# Open a connection to RabbitMQ on localhost using all default parameters
connection = pika.BlockingConnection(parameters=params)

# Open the channel
channel = connection.channel()

# Declare the queue
channel.queue_declare(
        callback=on_callback,
        queue="test",
        durable=True,
        exclusive=False,
        auto_delete=False
    )

# ...

# Re-declare the queue with passive flag
res = channel.queue_declare(
        callback=on_callback,
        queue="test",
        durable=True,
        exclusive=False,
        auto_delete=False,
        passive=True
    )
print 'Messages in queue %d' % res.method.message_count

This will print the following:

<Method(['frame_type=1', 'channel_number=1', "method=<Queue.DeclareOk(['queue=test', 'message_count=0', 'consumer_count=0'])>"])>
<Method(['frame_type=1', 'channel_number=1', "method=<Queue.DeclareOk(['queue=test', 'message_count=0', 'consumer_count=0'])>"])>
Messages in queue 0

You get the number of messages from the message_count member.

Answer 2

Have you tried PyRabbit? It has a get_queue_depth() method which sounds like what you're looking for.

Answer 3

There are two ways to get the queue size in the AMQP protocol. You can either use Queue.Declare or Basic.Get.

If you are consuming messages as they arrive using Basic.Consume, then you can't get this info unless you disconnect (timeout) and redeclare the queue, or else get one message but don't ack it. In newer versions of AMQP you can actively requeue the message.

As for Pika, I don't know the specifics but Python clients for AMQP have been a thorn in my side. Often you will need to monkeypatch classes in order to get the info you need, or to allow a queue consumer to timeout so that you can do other things at periodic intervals like record stats or find out how many messages are in a queue.

Another way around this is to give up, and use the Pipe class to run sudo rabbitmqctl list_queues -p my_vhost. Then parse the output to find the size of all queues. If you do this you will need to configure /etc/sudoers to not ask for the usual sudo password.

I pray that someone else with more Pika experience answers this by pointing out how you can do all the things that I mentioned, in which case I will download Pika and kick the tires. But if that doesn't happen and you are having difficulty with monkeypatching the Pika code, then have a look at haigha. I found their code to be much more straightforward than other Python AMQP client libraries because they stick closer to the AMQP protocol.

Answer 4

Here is how you can get queue length using pika(Considering you are using default user and password on localhost) replace q_name by your queue name.

import pika
connection = pika.BlockingConnection()
channel = connection.channel()
q = channel.queue_declare(q_name)
q_len = q.method.message_count