Celery single task persistent data

瘦欲@ 提交于 2019-12-12 15:15:58

问题


Lets say a single task is enough for a machine to stay very busy for a few minutes.

I want to get the result of the task, then depending on the result, have the worker perform the same task again.

The question I cannot find an answer to is this: Can I keep data in memory on the worker machine in order to use it on the next task?


回答1:


Yes you can. The documentation (http://docs.celeryproject.org/en/latest/userguide/tasks.html#instantiation) is a bit vague and I'm not sure if this is the best way, but you can do something like this:

This is what you'd like to do, but doesn't work:

# This doesn't work
a = 0
@celery.task
def mytask(x):
    a += x
    return a

This is how to make it work:

from celery import Task, registry
@celery.task
class MyTask(Task):
    def __init__(self):
        self.a = 0

    def run(self, x):
        self.a += x
        return self.a
mytask = registry.tasks[MyTask.name]

According to the docs:

A task is not instantiated for every request, but is registered in the task registry as a global instance. This means that the __init__ constructor will only be called once per process, and that the task class is semantically closer to an Actor. ... If you have a task, ... And you route every request to the same process, then it will keep state between requests."

I've had success with this but I don't know if there are better methods.



来源:https://stackoverflow.com/questions/17035087/celery-single-task-persistent-data

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!