Increase celery retry time each retry cycle

Question

I do retries with celery like in the Docs-Example:

@task()
def add(x, y):
    try:
        ...
    except Exception, exc:
        add.retry(exc=exc, countdow         


        

Answer 1

Here is a simple way to create bigger delay each time the task is evaluated. This value is updated by celery itself so you don't need to manage anything yourself.

@task()
def add(x, y):
    try:
        ...
    except Exception as exc:
        raise add.retry(exc=exc, countdown=60 * add.request.retries) 

Note: First task is repeated with countdown of 0. Because number of retries is 0 for the first run.

Answer 2

Keep a variable with your last retry time in it, and multiply it by 2 each time until it exceeds whatever level you want (or, keep a count if you prefer a certain number of times...)

Answer 3

Since version 4.2 you can use options autoretry_for and retry_backoff for this purposes, for example:

@task(max_retries=10, autoretry_for=(Exception,), retry_backoff=60)
def add(x, y):
    pass