Issues running airflow scheduler as a daemon process

前端 未结 2 1433
[愿得一人]
[愿得一人] 2021-02-04 08:08

I have an EC2 instance that is running airflow 1.8.0 using LocalExecutor. Per the docs I would have expected that one of the following two commands would have raise

2条回答
  •  无人共我
    2021-02-04 08:09

    Documentation might be dated?

    I normally start Airflow as following

    airflow kerberos -D
    airflow scheduler -D
    airflow webserver -D
    

    Here's airflow webeserver --help output (from version 1.8):

    -D, --daemon Daemonize instead of running in the foreground

    Notice there is not boolean flag possible there. Documentation has to be fixed.

    Quick note in case airflow scheduler -D fails:

    This is included in the comments, but it seems like it's worth mentioning here. When you run your airflow scheduler it will create the file $AIRFLOW_HOME/airflow-scheduler.pid. If you try to re-run the airflow scheduler daemon process this will almost certainly produce the file $AIRFLOW_HOME/airflow-scheduler.err which will tell you that lockfile.AlreadyLocked: /home/ubuntu/airflow/airflow-scheduler.pid is already locked. If your scheduler daemon is indeed out of commission and you find yourself needing to restart is execute the following commands:

    sudo rm $AIRFLOW_HOME airflow-scheduler.err  airflow-scheduler.pid
    airflow scheduler -D 
    

    This got my scheduler back on track.

提交回复
热议问题