Is there a way to pause a specific DagRun within Airflow?
I want to be able to have multiple, simultaneous executing runs of a single DAG, and I want to be able to pause
If this is about sensors, you are in luck because the solution was implemented in version 1.10.2: https://issues.apache.org/jira/browse/AIRFLOW-2747.
:param mode: How the sensor operates.
Options are: ``{ poke | reschedule }``, default is ``poke``.
When set to ``poke`` the sensor is taking up a worker slot for its
whole execution time and sleeps between pokes. Use this mode if the
expected runtime of the sensor is short or if a short poke interval
is requried.
When set to ``reschedule`` the sensor task frees the worker slot when
the criteria is not yet met and it's rescheduled at a later time. Use
this mode if the expected time until the criteria is met is. The poke
interval should be more than one minute to prevent too much load on
the scheduler.
Source: https://github.com/apache/airflow/blob/e62ad5333cbb56ae0f2001f0f79008a21c41a983/airflow/sensors/base_sensor_operator.py#L46