How to run one airflow task and all its dependencies?

我只是一个虾纸丫 提交于 2019-12-22 03:50:51

问题


I suspected that

airflow run dag_id task_id execution_date

would run all upstream tasks, but it does not. It will simply fail when it sees that not all dependent tasks are run. How can I run a specific task and all its dependencies? I am guessing this is not possible because of an airflow design decision, but is there a way to get around this?


回答1:


You can run a task independently by using -i/-I/-A flags along with the run command.

But yes the design of airflow does not permit running a specific task and all its dependencies.

You can backfill the dag by removing non-related tasks from the DAG for testing purpose




回答2:


A bit of a workaround but in case you have given your tasks task_id-s consistently you can try the backfilling from Airflow CLI (Command Line Interface):

airflow backfill -t TASK_REGEX ... dag_id

where TASK_REGEX corresponds to the naming pattern of the task you want to rerun and its dependencies.

(remember to add the rest of the command line options, like --start_date).



来源:https://stackoverflow.com/questions/42633892/how-to-run-one-airflow-task-and-all-its-dependencies

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!