Airflow - run task regardless of upstream success/fail

Question

I have a DAG which fans out to multiple independent units in parallel. This runs in AWS, so we have tasks which scale our AutoScalingGroup up to the maximum number of worker

Answer 1

All operators have an argument trigger_rule which can be set to 'all_done', which will trigger that task regardless of the failure or success of the previous task(s).

You could set the trigger rule for the task you want to run to 'all_done' instead of the default 'all_success'.

A simple bash operator task with that argument would look like:

task = BashOperator(
    task_id="hello_world",
    bash_command="echo Hello World!",
    trigger_rule="all_done",
    dag=dag
    )