How to replace environment variable value in yaml file to be parsed using python script

问题

I need to use environment variable "PATH" in yaml file which needs to be parsed with a script.

This is the environment variable I have set on my terminal:

$ echo $PATH
/Users/abc/Downloads/tbwork

This is my sample.yml:

---
Top: ${PATH}/my.txt
Vars:
- a
- b

When I parse this yaml file with my script, I don't see PATH variables actual value.

This is my script:

import yaml
import os
import sys

stream = open("sample.yml", "r")
docs = yaml.load_all(stream)
for doc in docs:
    for k,v in doc.items():
        print k, "->", v
    print "\n",

Output:

Top -> ${PATH}/my.txt
Vars -> ['a', 'b']

Expected output is:

Top -> /Users/abc/Downloads/tbwork/my.txt
Vars -> ['a', 'b']

Can someone help me figuring out the correct way to do it if I am doing it wrong way?

回答1:

PY-yaml library doesn't resolve environment variables by default. You need to define an implicit resolver that will find the regex that defines an environment variable and execute a function to resolve it.

You can do it through yaml.add_implicit_resolver and yaml.add_constructor. In the code below, you are defining a resolver that will match on ${ env variable } in the YAML value and calling the function path_constructor to look up the environment variable.

import yaml
import re
import os

path_matcher = re.compile(r'\$\{([^}^{]+)\}')
def path_constructor(loader, node):
  ''' Extract the matched value, expand env variable, and replace the match '''
  value = node.value
  match = path_matcher.match(value)
  env_var = match.group()[2:-1]
  return os.environ.get(env_var) + value[match.end():]

yaml.add_implicit_resolver('!path', path_matcher)
yaml.add_constructor('!path', path_constructor)

data = """
env: ${VAR}/file.txt
other: file.txt
"""

if __name__ == '__main__':
  p = yaml.load(data, Loader=yaml.FullLoader)
  print(os.environ.get('VAR')) ## /home/abc
  print(p['env']) ## /home/abc/file.txt

Warning: Do not run this if you are not the one specifying the env variables (or any other untrusted input) as there are remote code execution vulnerabilities with FullLoader as of July 2020.

回答2:

Here is an alternative version which does use a new Loader class if you do not want to modify the global/default yaml Loader.

And more importantly, it correctly replaces interpolated strings that are not just the environment variables, eg path/to/${SOME_VAR}/and/${NEXT_VAR}/foo/bar

        path_matcher = re.compile(r'.*\$\{([^}^{]+)\}.*')
        def path_constructor(loader, node):
            return os.path.expandvars(node.value)

        class EnvVarLoader(yaml.SafeLoader):
            pass

        EnvVarLoader.add_implicit_resolver('!path', path_matcher, None)
        EnvVarLoader.add_constructor('!path', path_constructor)

        with open(configPath) as f:
            c = yaml.load(f, Loader=EnvVarLoader)

回答3:

There is a nice library envyaml for this. With it it's very simple:

from envyaml import EnvYAML

# read file env.yaml and parse config
env = EnvYAML('env.yaml')

回答4:

Using yamls add_implicit_resolver and add_constructor works for me but like this with the above example:

import yaml
import re
import os
os.environ['VAR']="you better work"
path_matcher = re.compile(r'\$\{([^}^{]+)\}')
def path_constructor(loader, node):

  ''' Extract the matched value, expand env variable, and replace the match '''
  print("i'm here")
  value = node.value
  match = path_matcher.match(value)
  env_var = match.group()[2:-1]
  return os.environ.get(env_var) + value[match.end():]

yaml.add_implicit_resolver('!path', path_matcher, None, yaml.SafeLoader)
yaml.add_constructor('!path', path_constructor, yaml.SafeLoader)

data = """
env: ${VAR}/file.txt
other: file.txt
"""

if __name__ == '__main__':
  p = yaml.safe_load(data)
  print(os.environ.get('VAR')) ## you better work
  print(p['env']) ## you better work/file.txt

来源：https://stackoverflow.com/questions/52412297/how-to-replace-environment-variable-value-in-yaml-file-to-be-parsed-using-python

标签

python

python-2.7

pyyaml