How to efficiently do many tasks a “little later” in Python?

Question

I have a process, that needs to perform a bunch of actions \"later\" (after 10-60 seconds usually). The problem is that those \"later\" actions can be a lot (1000s), so using a

Answer 1

If you have a bunch of tasks that need to get performed later, and you want them to persist even if you shut down the calling program or your workers, you should really look into Celery, which makes it super easy to create new tasks, have them executed on any machine you'd like, and wait for the results.

From the Celery page, "This is a simple task adding two numbers:"

from celery.task import task

@task
def add(x, y):
    return x + y

You can execute the task in the background, or wait for it to finish:

>>> result = add.delay(8, 8)
>>> result.wait() # wait for and return the result
16