How to set/get airflow variables which are in json format from command line

问题

I can't edit values of airflow variables in json format through cloud shell.

I am using cloud shell to access my airflow variable params (in json format) and it gives me the complete json when i use following command:

gcloud composer environments run composer001 
--location us-east1 variables 
--get params

However I want to edit one of the values inside json, how do i access that?

I referred to the documentation and various other links on google however could only find how to set variables that are not in json format but are a single value variables.

回答1:

Cloud Composer CLI and Airflow CLI only operate on top-level variables, not their JSON contents.

You can use Airflow UI to edit your JSON variable, as the UI loads the whole variable and you can edit it in place. Or if you need to update a specific value inside your JSON variable through command line, you can first export your variables to a JSON file:

gcloud composer environments run \
    [ENVIRONMENT] --location [LOCATION] \
    variables -- --export /home/airflow/gcs/data/your-vars.json

gcloud composer environments storage data export \
    --environment [ENVIRONMENT] --location [LOCATION] \
    --source your-vars.json --destination .

edit the value inside JSON using a command like jq:

jq '.params.jsonkey = "newvalue"' your-vars.json > your-updated-vars.json

and import the updated file back to Cloud Composer:

gcloud composer environments storage data import \
    --environment [ENVIRONMENT] --location [LOCATION] \
    --source your-updated-vars.json

gcloud composer environments run \
    [ENVIRONMENT] --location [LOCATION] \
    variables -- --import /home/airflow/gcs/data/your-updated-vars.json

来源：https://stackoverflow.com/questions/57103465/how-to-set-get-airflow-variables-which-are-in-json-format-from-command-line

标签

google-cloud-platform

google-cloud-composer