Accessing Kubernetes Secret from Airflow KubernetesPodOperator

问题

I'm setting up an Airflow environment on Google Cloud Composer for testing. I've added some secrets to my namespace, and they show up fine:

$ kubectl describe secrets/eric-env-vars
Name:         eric-env-vars
Namespace:    eric-dev
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
VERSION_NUMBER:  6 bytes

I've referenced this secret in my DAG definition file (leaving out some code for brevity):

env_var_secret = Secret(
    deploy_type='env',
    deploy_target='VERSION_NUMBER',
    secret='eric-env-vars',
    key='VERSION_NUMBER',
)

dag = DAG('env_test', schedule_interval=None, start_date=start_date)

operator = KubernetesPodOperator(
    name='k8s-env-var-test',
    task_id='k8s-env-var-test',
    dag=dag,
    image='ubuntu:16.04',
    cmds=['bash', '-cx'],
    arguments=['env'],
    config_file=os.environ['KUBECONFIG'],
    namespace='eric-dev',
    secrets=[env_var_secret],
)

But when I run this DAG, the VERSION_NUMBER env var isn't printed out. It doesn't look like it's being properly linked to the pod either (apologies for imprecise language, I am new to both Kubernetes and Airflow). This is from the Airflow task log of the pod creation response (also formatted for brevity/readability):

'env': [
{
    'name': 'VERSION_NUMBER',
    'value': None,
    'value_from': {
        'config_map_key_ref': None,
        'field_ref': None,
        'resource_field_ref': None,
        'secret_key_ref': {
            'key': 'VERSION_NUMBER',
            'name': 'eric-env-vars',
            'optional': None}
        }
    }
]

I'm assuming that we're somehow calling the constructor for the Secret wrong, but I am not entirely sure. Guidance appreciated!

回答1:

Turns out this was a misunderstanding of the logs!

When providing an environment variable to a Kubernetes pod via a Secret, that value key in the API response is None because the value comes from the secret_key_ref.

来源：https://stackoverflow.com/questions/55303229/accessing-kubernetes-secret-from-airflow-kubernetespodoperator

标签

Kubernetes

airflow

google-cloud-composer