DAG is not visible on Airflow UI

自闭症网瘾萝莉.ら 提交于 2020-06-17 09:34:28

问题


This is my dag file in dags folder.

Code that goes along with the Airflow located at:
http://airflow.readthedocs.org/en/latest/tutorial.html
"""
from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.python_operator import PythonOperator
from datetime import datetime, timedelta
from work_file import Test


class Main(Test):
    def __init__(self):
        super(Test, self).__init__()

    def create_dag(self):
        default_args = {
            "owner": "airflow",
            "depends_on_past": False,
            "start_date": datetime(2015, 6, 1),
            "email": ["airflow@airflow.com"],
            "email_on_failure": False,
            "email_on_retry": False,
            "retries": 1,
            "retry_delay": timedelta(minutes=5),
            # 'queue': 'bash_queue',
            # 'pool': 'backfill',
            # 'priority_weight': 10,
            # 'end_date': datetime(2016, 1, 1),
        }
        dag = DAG("python_dag", default_args=default_args, schedule_interval='0 * * * *')

        dummy_task = DummyOperator(task_id='dummy_task', retries=3)

        python_task = PythonOperator(task_id='python_task', python_callable=self.my_func)

        dummy_task >> python_task


if __name__ == "__main__":
    a = Main()
    a.create_dag()

This is my other file work_file.py which is in the same dags folder.

class Test:
    def __init__(self):
        pass

    def my_func(self):
        return "Hello"

Aim:-The Aim is to call the my_func from my dag file.

Problem:- There seems to be no error on the UI,but my dag python_dag is not visible.

My server, scheduler is also running, I've tried restarting the same but nothing happened.

I have imported the file as well (from work_file import Test)

Thanks in advance!


回答1:


There are multiple problems with the DAG:

  1. The operators are not assigned to any DAG. Add dag=dag to the constructors. E.g., DummyOperator(..., dag=dag).
  2. create_dag() does not return the DAG. Add return dag.
  3. The DAG script is not executed as the top level code. That is, the modules __name__ is never '__main__'. Remove if __name__ == "__main__":.
  4. The DAG objects must be in the global namespace of the module. Assign the return value of create_dag() to a variable: dag = a.create_dag().


来源:https://stackoverflow.com/questions/62327923/dag-is-not-visible-on-airflow-ui

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!