Celery groups and chains

问题

I need to sort some tasks in Celery that some of them should as a single task and some should work parallel and when the tasks in the group completed, it should pass the next one:

chain(
    task1.s(),
    task2.s(),
    group(task3.s(), task4.s()), 
    group(task5.s(), task6.s(), task7.s()), 
    task7.s()
).delay()

But I think what did I do is wrong. Any body have idea how to do it?

Also, I don't care about sending the result of each task to the others.

回答1:

This sounds like a chord, ie where you execute tasks in parallel and have a callback into another task when the parallel tasks are finished: http://docs.celeryproject.org/en/latest/userguide/canvas.html#chords

So you might have to change it something like: chain(task1.s(), task2.s(), chord(task3.s(), task4.s())(chord(task5.s(), task6.s(), task7.s())(task7.s())))

Also, chains/groups etc always return the results and pass them on to the child task(s) so you have to model the task arguments accordingly.

As it's quite a complex workflow, you might be better off calling the next task from with the previous task (like calling task2.s().delay() at the end of task1) - but I guess there's no way around modelling the chord.

回答2:

This one finally worked:

chain(
    task1.s(),
    task2.s(),
    chord([task3.s(), task4.s()], body=task_result.s(), immutable=True), 
    chord([task5.s(), task6.s(), task7.s()], body=task_result.s(), immutable=True), 
    task7.s()
).delay()

来源：https://stackoverflow.com/questions/51076827/celery-groups-and-chains