Is it possible for Airflow scheduler to first finish the previous day's cycle before starting the next?

前端 未结 3 871
暗喜
暗喜 2021-02-02 16:50

Right now, nodes in my DAG proceeds to the next day\'s task before the rest of the nodes of that DAG finishes. Is there a way for it to wait for the rest of the DAG to finish be

3条回答
  •  你的背包
    2021-02-02 17:22

    What ended up working for me is a combination of

    1. Adding task dependencies : wait_for_downstream=True, depends_on_past=True
    2. Adding max_active_runs:1 to while creating the dag. I did try to add max_active_runs as a default argument, but that did not work.

提交回复
热议问题