When does a airflow dag definition get evaluated?

前端 未结 2 1190
旧巷少年郎
旧巷少年郎 2021-01-24 02:08

Suppose I have an airflow dag file that creates a graph like so...

def get_current_info(filename)
    current_info = {}
           


        
相关标签:
2条回答
  • 2021-01-24 02:33

    After some discussion on the [airflow email list][1], it turns out that airflow builds the dag for each task when it is run (so each tasks includes the overhead of building the dag again (which in my case was very significant)).

    See more details on this here: https://stackoverflow.com/a/59995882/8236733

    0 讨论(0)
  • 2021-01-24 02:46

    The DAGs are evaluated in every run of the scheduler.

    This article describes how the scheduler works and at what stage the DAG files are picked up for evaluation.

    0 讨论(0)
提交回复
热议问题