How to set request_cpu globally for airflow worker pods using the kubernetes executor?

Question

I\'m trying to set the request_cpu parameter in the Kubernetes executor for Airflow but haven\'t been able to find where I can do that. In the default airflow confi

Answer 1

You can set this executor_config at task level for KubernetesExecutor as follows.

    exmaple_task = PythonOperator(
        task_id="exmaple_task",
        python_callable=print_stuff,
        executor_config={
            "KubernetesExecutor": {"request_cpu": "1",
                                   "request_memory": "128Mi",
                                   "limit_memory": "128Mi"}}
    )

You can define the following in executor_config :

• image
• image_pull_policy
• request_memory
• request_cpu
• limit_memory
• limit_cpu
• limit_gpu
• node_selectors
• affinity
• tolerations
• annotations
• volumes
• volume_mounts
• volume_mounts

Doc: https://airflow.apache.org/docs/1.10.9/_api/airflow/contrib/executors/kubernetes_executor/index.html#airflow.contrib.executors.kubernetes_executor.KubernetesExecutorConfig