Celery: auto discovery does not find tasks module in app

问题

I have the following setup with a fresh installed celery and django 1.4:

settings.py:

import djcelery
djcelery.setup_loader()

BROKER_HOST = 'localhost'
BROKER_PORT = 5672
BROKER_USER = 'user'
BROKER_PASSWORD = 'password'
BROKER_VHOST = 'test'

[...]

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.admin',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.staticfiles',
    'djcelery',
    'south',
    'compressor',
    'testapp',
]

testapp/tasks.py:

from celery.task import task

@task()
def add(x, y):
    return x + y

Message delivery to the celeryd works fine, but the task is always unregistered (so auto discovery does not seem to work correctly here). Only if I import the tasks module in tasks/__init__.py the task is found and I can use it.

Also the documentation was a little confusing about the decorator import, but I think this is the right one now.

Where is the bug in my setup?

回答1:

This was a bug in django-celery 2.5.4, please upgrade to 2.5.5!

回答2:

Add CELERY_IMPORTS to your settings.py:

CELERY_IMPORTS = ('testapp.tasks',)

Import all the tasks in testapp.tasks.__init__ file

Then Celery will import all tasks from testapp.tasks folder and name them as they are

回答3:

For any one who stumbles here looking for similar problem solution.

In my case it was switching from old module bases INSTALLED_APPS setting to a new AppConfig based configuration.

New applications should avoid default_app_config. Instead they should require the dotted path to the appropriate AppConfig subclass to be configured explicitly in INSTALLED_APPS.

To fix this you should change the way you feed packages to celery, as stated here in the 2248 Celery issue:

from django.apps import apps
app.autodiscover_tasks(lambda: [n.name for n in apps.get_app_configs()]

Instead of the old Celery 3 way:

app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

回答4:

I had the same issue with django 1.4.1 celery 3.0.9 and fixed it by naming the task.

@task() -> @task(name='testapp.tasks.add')

回答5:

I'm pretty sure you have to import the "Celery app instance" and declare a task like this:

from project_name.celery import app

@app.task
def video_process_task(video_id):
    pass

note that there's a celery.py file in the project_dir/project_name folder, which declares the celery instance, like this:

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project_name.settings')
app = Celery('project_name')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

as instructed in the official documentation.

also, you must register the task, in one of these three ways:

the CELERY_IMPORTS variable in Django project's settings.py file as in dgel's answer
passing bind=True to the decorator like: @app.task(bind=True)
if you set autodiscovery for celery like it is done above in the line app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) , each Django application's folder can contain a tasks.py file, and the tasks declared inside it will be automatically registered

note that changes to the tasks REQUIRE A CELERY RESTART to take effect

回答6:

In my case, I couldn't figure out the problem until I tried to import the tasks in shell (python or python manage.py shell).

>>> from project_name.tasks import task_add

来源：https://stackoverflow.com/questions/10236318/celery-auto-discovery-does-not-find-tasks-module-in-app

标签

python

django

celery

django-celery