Question

我需要在yaml文件中使用环境变量“ PATH”，该环境变量需要使用脚本进行解析。

这是我在终端上设置的环境变量：

$ echo $PATH
/Users/abc/Downloads/tbwork

这是我的sample.yml：

---
Top: ${PATH}/my.txt
Vars:
- a
- b

当我使用脚本解析该Yaml文件时，看不到PATH变量的实际值。

这是我的脚本：

import yaml
import os
import sys

stream = open("sample.yml", "r")
docs = yaml.load_all(stream)
for doc in docs:
    for k,v in doc.items():
        print k, "->", v
    print "\n",

输出：

Top -> ${PATH}/my.txt
Vars -> ['a', 'b']

预期输出为：

Top -> /Users/abc/Downloads/tbwork/my.txt
Vars -> ['a', 'b']

如果我做错了方向，有人可以帮助我找出正确的方法吗？

Answer 1

PY-yaml库默认情况下不解析环境变量。您需要定义一个隐式解析器，该解析器将找到定义环境变量的正则表达式并执行函数来对其进行解析。

您可以通过yaml.add_implicit_resolver和yaml.add_constructor进行操作。在下面的代码中，您将定义一个解析器，该解析器将与YAML值中的$ {env变量}匹配，并调用函数path_constructor来查找环境变量。

import yaml
import re
import os

path_matcher = re.compile(r'\$\{([^}^{]+)\}')
def path_constructor(loader, node):
  ''' Extract the matched value, expand env variable, and replace the match '''
  value = node.value
  match = path_matcher.match(value)
  env_var = match.group()[2:-1]
  return os.environ.get(env_var) + value[match.end():]

yaml.add_implicit_resolver('!path', path_matcher)
yaml.add_constructor('!path', path_constructor)

data = """
env: ${VAR}/file.txt
other: file.txt
"""

if __name__ == '__main__':
  p = yaml.safe_load(data)
  print(os.environ.get('VAR')) ## /home/abc
  print(p['env']) ## /home/abc/file.txt

Answer 2

有一个不错的库envyaml。有了它非常简单：

from envyaml import EnvYAML

# read file env.yaml and parse config
env = EnvYAML('env.yaml')

Answer 3

您可以看到如何here，这导致了非常小的库pyaml-env 以便于使用，这样我们就不会在每个项目中重复事情。

因此，使用该库，您的示例 yaml 将变为：

---
Top: !ENV ${PATH}/my.txt
Vars:
- a
- b

和parse_config

from pyaml_env import parse_config
config = parse_config('path/to/config.yaml')

print(config)
# outputs the following, with the environment variables resolved
{
    'Top': '/Users/abc/Downloads/tbwork/my.txt'
    'Vars': ['a', 'b']
}

如果您愿意，也可以选择使用默认值，如下所示：

---
Top: !ENV ${PATH:'~/data/'}/my.txt
Vars:
- a
- b

关于实现，简而言之：为了让 PyYAML 能够解析环境变量，我们需要做三件事：

环境变量标识的正则表达式模式，例如pattern = re.compile('.?${(\w+)}.?')
表示有一个（或更多）环境变量需要解析的标签，例如!ENV。
以及加载器将用来解析环境变量的函数

一个完整的例子：

import os
import re
import yaml


def parse_config(path=None, data=None, tag='!ENV'):
    """
    Load a yaml configuration file and resolve any environment variables
    The environment variables must have !ENV before them and be in this format
    to be parsed: ${VAR_NAME}.
    E.g.:
    database:
        host: !ENV ${HOST}
        port: !ENV ${PORT}
    app:
        log_path: !ENV '/var/${LOG_PATH}'
        something_else: !ENV '${AWESOME_ENV_VAR}/var/${A_SECOND_AWESOME_VAR}'
    :param str path: the path to the yaml file
    :param str data: the yaml data itself as a stream
    :param str tag: the tag to look for
    :return: the dict configuration
    :rtype: dict[str, T]
    """
    # pattern for global vars: look for ${word}
    pattern = re.compile('.*?\${(\w+)}.*?')
    loader = yaml.SafeLoader

    # the tag will be used to mark where to start searching for the pattern
    # e.g. somekey: !ENV somestring${MYENVVAR}blah blah blah
    loader.add_implicit_resolver(tag, pattern, None)

    def constructor_env_variables(loader, node):
        """
        Extracts the environment variable from the node's value
        :param yaml.Loader loader: the yaml loader
        :param node: the current node in the yaml
        :return: the parsed string that contains the value of the environment
        variable
        """
        value = loader.construct_scalar(node)
        match = pattern.findall(value)  # to find all env variables in line
        if match:
            full_value = value
            for g in match:
                full_value = full_value.replace(
                    f'${{{g}}}', os.environ.get(g, g)
                )
            return full_value
        return value

    loader.add_constructor(tag, constructor_env_variables)

    if path:
        with open(path) as conf_data:
            return yaml.load(conf_data, Loader=loader)
    elif data:
        return yaml.load(data, Loader=loader)
    else:
        raise ValueError('Either a path or data should be defined as input')

Answer 4

这里是替代版本，如果您不想修改全局/默认yaml Loader，它会使用新的Loader类。

更重要的是，它可以正确替换不只是环境变量的内插字符串，例如path/to/${SOME_VAR}/and/${NEXT_VAR}/foo/bar

        path_matcher = re.compile(r'.*\$\{([^}^{]+)\}.*')
        def path_constructor(loader, node):
            return os.path.expandvars(node.value)

        class EnvVarLoader(yaml.SafeLoader):
            pass

        EnvVarLoader.add_implicit_resolver('!path', path_matcher, None)
        EnvVarLoader.add_constructor('!path', path_constructor)

        with open(configPath) as f:
            c = yaml.load(f, Loader=EnvVarLoader)

Answer 5

使用yamls add_implicit_resolver和add_constructor对我有用，但在上面的示例中是这样的：

import yaml
import re
import os
os.environ['VAR']="you better work"
path_matcher = re.compile(r'\$\{([^}^{]+)\}')
def path_constructor(loader, node):

  ''' Extract the matched value, expand env variable, and replace the match '''
  print("i'm here")
  value = node.value
  match = path_matcher.match(value)
  env_var = match.group()[2:-1]
  return os.environ.get(env_var) + value[match.end():]

yaml.add_implicit_resolver('!path', path_matcher, None, yaml.SafeLoader)
yaml.add_constructor('!path', path_constructor, yaml.SafeLoader)

data = """
env: ${VAR}/file.txt
other: file.txt
"""

if __name__ == '__main__':
  p = yaml.safe_load(data)
  print(os.environ.get('VAR')) ## you better work
  print(p['env']) ## you better work/file.txt

如何在要使用python脚本解析的yaml文件中替换环境变量值

5 个答案: