Is there a way to pause an airflow DagRun?

后端 未结 1 841
梦谈多话
梦谈多话 2021-01-25 09:04

Is there a way to pause a specific DagRun within Airflow?

I want to be able to have multiple, simultaneous executing runs of a single DAG, and I want to be able to pause

相关标签:
1条回答
  • 2021-01-25 09:44

    If this is about sensors, you are in luck because the solution was implemented in version 1.10.2: https://issues.apache.org/jira/browse/AIRFLOW-2747.

    :param mode: How the sensor operates.
        Options are: ``{ poke | reschedule }``, default is ``poke``.
        When set to ``poke`` the sensor is taking up a worker slot for its
        whole execution time and sleeps between pokes. Use this mode if the
        expected runtime of the sensor is short or if a short poke interval
        is requried.
        When set to ``reschedule`` the sensor task frees the worker slot when
        the criteria is not yet met and it's rescheduled at a later time. Use
        this mode if the expected time until the criteria is met is. The poke
        interval should be more than one minute to prevent too much load on
        the scheduler.
    

    Source: https://github.com/apache/airflow/blob/e62ad5333cbb56ae0f2001f0f79008a21c41a983/airflow/sensors/base_sensor_operator.py#L46

    0 讨论(0)
提交回复
热议问题