Airflow Task failure/retry workflow

Question

I have retry logic for tasks and it\'s not clear how Airflow handles task failures when retries are turned on.

Their documentation just states that on_failure_

Answer 1

Retry logic/parameters will take place before failure logic/parameters. So if you have a task set to retry twice, it will attempt to run again two times (and thus executing on_retry_callback ) before failing (and then executing on_failure_callback).

An easy way to confirm the sequence that it is executed in is to set your email_on_retry and email_on_failure to True and see the order in which they appear. You can physically confirm that it will retry before failing.

default_args = {
    'owner': 'me',
    'start_date': datetime(2019, 2, 8),
    'email': ['you@work.com'],
    'email_on_failure': True,
    'email_on_retry': True,
    'retries': 1,
    'retry_delay': timedelta(minutes=1)
}