How to use priority in celery task.apply_async

Question

I have a test queue in celery and I have defined a task for it:

@celery_app.task(queue=\'test\', ignore_result=True)
def priority_test(priority):
           


        

Answer 1

In order to have priority working properly you need to properly configure a couple of settings and you need at least version 3.5.0 of RabbitMQ.

First set the x-max-priority of your queue to 10. From the docs :

from kombu import Exchange, Queue

app.conf.task_queues = [
    Queue('tasks', Exchange('tasks'), routing_key='tasks',
          queue_arguments={'x-max-priority': 10},
]

A default value for all queues can be set using the task_queue_max_priority setting:

app.conf.task_queue_max_priority = 10

Then configure the following settings:

CELERY_ACKS_LATE = True
CELERYD_PREFETCH_MULTIPLIER = 1

By default the prefetch multiplier is 4, which in your case will cause the first 4 tasks with priority 10, 9, 8 and 7 to be fetched before the other tasks are present in the queue. The CELERY_ACKS_LATE setting will cause the tasks to be acknowledged after they have been executed. You can experiment with this setting to see what behaviour you prefer.