Airflow: Why is there a start_date for operators?

前端 未结 4 1848
北海茫月
北海茫月 2021-01-04 23:24

I don\'t understand why do we need a \'start_date\' for the operators(task instances). Shouldn\'t the one that we pass to the DAG suffice?

Also, if the current time

4条回答
  •  执笔经年
    2021-01-05 00:25

    Just to add to what is already here. A task that depends on another task(s) must have a start date >= to the start date of its dependencies.

    For example:

    • if task_a depends on task_b
    • you cannot have
      • task_a start_date = 1/1/2019
      • task_b start_date = 1/2/2019
      • Otherwise, task_a will not be runnable for 1/1/2019 as task_b will not run for that date and you cannot mark it as complete either

    Why would you want this?

    • I would have liked this logic for a task, which was an external task sensor waiting for the completion of another dag. But the other dag had a start date after the current dag. Therefore, I didn't want the dependency in place for days when the other dag didn't exist

提交回复
热议问题