airflow-scheduler

来源： https://stackoverflow.com/questions/58101482/airflow-externaltasksensor-stuck

来源： https://stackoverflow.com/questions/63420667/can-airflow-persist-access-to-metadata-of-short-lived-dynamically-generated-task

来源： https://stackoverflow.com/questions/62962386/can-an-airflow-task-dynamically-generate-a-dag-at-runtime

来源： https://stackoverflow.com/questions/62962386/can-an-airflow-task-dynamically-generate-a-dag-at-runtime

来源： https://stackoverflow.com/questions/54665041/how-to-persist-a-value-during-airflow-retry

来源： https://stackoverflow.com/questions/54665041/how-to-persist-a-value-during-airflow-retry

来源： https://stackoverflow.com/questions/63527204/airflow-how-to-get-all-the-future-run-date

问题 I am using apache airflow for running my dags. I am getting an exception as: *** Log file does not exist: /opt/airflow/logs/download2/download2/2020-07-26T15:00:00+00:00/1.log *** Fetching from: http://fb3393f5f01e:8793/log/download2/download2/2020-07-26T15:00:00+00:00/1.log *** Failed to fetch log file from worker. HTTPConnectionPool(host='fb3393f5f01e', port=8793): Max retries exceeded with url: /log/download2/download2/2020-07-26T15:00:00+00:00/1.log (Caused by NewConnectionError('<urllib3

问题 I am using apache airflow for running my dags. I am getting an exception as: *** Log file does not exist: /opt/airflow/logs/download2/download2/2020-07-26T15:00:00+00:00/1.log *** Fetching from: http://fb3393f5f01e:8793/log/download2/download2/2020-07-26T15:00:00+00:00/1.log *** Failed to fetch log file from worker. HTTPConnectionPool(host='fb3393f5f01e', port=8793): Max retries exceeded with url: /log/download2/download2/2020-07-26T15:00:00+00:00/1.log (Caused by NewConnectionError('<urllib3

问题 I am trying to run a DAG for every 5 minutes starting from today(2019-12-18). I defined my start date as start_date:dt.datetime(2019, 12, 18, 10, 00, 00) and schedule interval as schedule_interval= '*/5 * * * *' . When I start the airflow scheduler I don't see any of my tasks running. But when I modify the start_date as start_date:dt.datetime(2019, 12, 17, 10, 00, 00) i.e., Yesterdays date, the DAG runs continuously like for every 10 seconds but not 5 minutes. I think the solution to this