Run DAG with tasks of different interval

大城市里の小女人 提交于 2019-12-11 18:19:04

问题


I have 3 tasks, A, B and C. I want to run task A only once, and then run task B monthly until end_date, then run task C only once to clean up.

This is similar to this question, but not applicable. How to handle different task intervals on a single Dag in airflow?

Thanks for your help


回答1:


For task A that is supposed to run only once, you can take inspiration from here


As far as tasks B & C are concerned, they can be tied up with A using a ShortCircuitOperator (as already told in the link you cited)

                   -> B
                 /
A -> ShortCircuit
                 \
                   -> C

Alternatively, you could skip B and C internally using an AirflowSkipException



来源:https://stackoverflow.com/questions/57060047/run-dag-with-tasks-of-different-interval

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!