We are using celery for our asynchronous background tasks and we have 2 queues for different priority tasks. We have 2 cluster of nodes serving them separately. Things are worki
Unfortunately, it is not possible with celery out of the box.
An optimum solution is to start 2 workers. 1 for low priority and other for high priority with n process.
When there are no high priority tasks, the worker with low priority tasks will use all the resources and vice versa. If there are both tasks, resources will be evenly distributed.