Celery: ignoring BROKER_URL in config file

Question

My question is a duplicate of this one, but more detailed.

The problem is that I have a BROKER_URL set in my Celery config file, but that isn't reflected in and I am loading the config: I checked, and it is being loaded - in fact, other constants defined there are being set, just not BROKER_URL.

This appears to be a bug, but I wanted to be sure.

---

celeryconfig.py:

BROKER_URL = "amqp://user:password@remote.server.com:5672//vhost"

CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = True

(JSON is being used as the serializer, not Pickle, so I know this is working.)

app.py:

from celery import Celery

app = Celery('tasks', broker='amqp://guest@localhost//')
app.config_from_object('celeryconfig')

Invoking the Worker:

celery -A app.app worker -l info

But then I get this:

[2013-11-12 11:20:51,610: INFO/MainProcess] consumer: Connected to amqp://guest@127.0.0.1:5672//.

---

I tried breaking up BROKER_URL, but to no avail:

BROKER_TRANSPORT = 'amqp'
BROKER_USER = 'user'
BROKER_PASSWORD = 'password'
BROKER_HOST = 'remote.server.com'
BROKER_PORT = 5672
BROKER_VHOST = '/vhost'

Interestingly, it does work when I explicitly set the BROKER_URL in app.py:

from celery import Celery

app = Celery('tasks', broker='amqp://guest@localhost//')
app.config_from_object('celeryconfig')
app.conf.BROKER_URL = "amqp://user:password@remote.server.com:5672//vhost"

Answer 1

Of course, I realized what I'd done wrong immediately after finishing this question, but I still posted it because someone might find it useful.

My problem is that I copied and pasted code from the tutorial (*facepalm).

I'm overriding the config file when I define the app with a broker arg:

app = Celery('tasks', broker='amqp://guest@localhost//')

Simply remove that:

app = Celery('tasks')

Tada! Everything works fine... and I learned a valuable lesson.

Answer 2

Just for clarification because you are using this:

app.config_from_object('django.conf:settings', namespace='CELERY')

in your celeryconfig.py:

CELERY_BROKER_URL = "amqp://user:password@remote.server.com:5672//vhost"

CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = True

Answer 3

Just for anyone else with a similar issue, I ran into a similar problem with a different cause.

I'm using celery with django on heroku, but in my dev environment I am using django-environ to populate environment variables from a file.

I forgot to preface my celery worker command with DJANGO_READ_DOT_ENV_FILE=1

So I went from:

celery -A myapp worker -l info

to

DJANGO_READ_DOT_ENV_FILE=1 celery -A myapp worker -l info

Silly, but a good reminder. Hope it saves someone some time.