Airflow: changing the crontab time for a DAG in Airflow

Question

I have a DAG that has been running everyday at 3:00, it ran OK for the past few weeks.

I\'ve updated the date to run now at 7:00, but apparently for the last 2 days

Answer 1

You can use the same dag. After modifying schedule_interval, you need to mark the previous job as succeeded via airflow backfill -m command.

Answer 2

To schedule a dag, Airflow just looks for the last execution date and sum the schedule interval. If this time has expired it will run the dag. You cannot simple update the start date. A simple way to do this is edit your start date and schedule interval, rename your dag (e.g. xxxx_v2.py) and redeploy it.

Answer 3

David,
1. You can also delete the dag via Experimental REST API. deleting a DAG
2. Change the desired start_date.
3. And add the same DAG back.

Answer 4

An alternative solution to renaming the DAG is to edit the execution_date of all prior task instances and DAG runs of the DAG in the database. The tables to alter are task_instance and dag_run respectively.

One of the downsides of this approach is that you will lose the ability to browse logs of completed tasks through the webserver.