Create dynamic queues with Celery

Question

Here\'s my scenario:

When a user logs in to my website, I queue up a bunch of tasks for the given user (typically each task takes 100s of msecs and there are 100s of tas

Answer 1

http://docs.celeryproject.org/en/latest/userguide/workers.html#queues

celery -A proj control add_consumer foo -d worker1.local

The same can be accomplished dynamically using the app.control.add_consumer() method:

app.control.add_consumer('foo', reply=True)
[{u'worker1.local': {u'ok': u"already consuming from u'foo'"}}]

app.control.add_consumer('foo', reply=True,
destination=['worker1@example.com'])

Answer 2

You can dynamically assign a task to a Queue at runtime when invoking it refer calling.html#routing-options. This would work if you have CELERY_CREATE_MISSING_QUEUES enabled.