How can I get execution_date in dag?? the outside of operator?

对着背影说爱祢 提交于 2020-07-08 12:26:50

问题


How can I get an execution_date parameter in outside of dag?

execution_min = "{{execution_date.strftime('%M') }}"

if execution_min == '00':
    logging.info('**** ' + "YES, It's 00")
    final_task = DummyOperator(
        task_id='task_y00',
        ...
        dag=dag
    )
else:
    logging.info('**** ' + "NOPE!!!")
    final_task = DummyOperator(
        task_id='task_n00',
        ...
        dag=dag
    )

I want to set a task stream with dynamically with execution_date (especially minute)

But Jinja template won't work with template_fields = ['execution_date']

Are there any solutions to get the execution parameter from outside of operator (= in the DAG itself) ???


回答1:


Execution date is specific to a DagRun. DagRun information is not available in a DAG definition file (it is available in an Operator's template fields because these get parsed at run-time via Jinja). DAG definition files are parsed frequently by the scheduler, webserver and workers even when the dag isn't running. This is why outside of an actual DagRun there is no access to things like execution date.

Furthermore there is no way to add/subtract tasks to a DAG run at runtime. You can have dynamic dags whose structure is decided before they run (ie. parse a file into a DAG structure), but you can't add tasks or decide what a DAG looks like while it's running.




回答2:


Try to just use execution_min = "{{ execution_date }}" then strftime after, make sure you have space before and after double braces.

UPDATE: If you are using it outside of the Operator, it won't work, you can pass a kwargs in then use it. Airflow: pass {{ ds }} as param to PostgresOperator



来源:https://stackoverflow.com/questions/46808531/how-can-i-get-execution-date-in-dag-the-outside-of-operator

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!