如何使用yml文件创建conda环境而不会出现此错误?

时间:2019-08-06 18:02:05

标签: python docker apache-spark conda

我正在按照here教程在AWS上进行PySpark。

我的操作系统:macOS High Sierra 10.12.6

到目前为止,一切都按教程中的方式进行。

我已经成功创建了“ hello-spark.yml”文件,并以精美的文字打开了文件,编辑的部分也在那里。

我运行以下代码时收到错误消息: conda env create -f hello-spark.yml

myComputerName anaconda3 MyPersonalName$ conda env create -f hello-spark.yml

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda/exceptions.py", line 1043, in __call__
        return func(*args, **kwargs)
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/cli/main.py", line 73, in do_call
        exit_code = getattr(module, func_name)(args, parser)
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/cli/main_create.py", line 77, in execute
        directory=os.getcwd())
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/specs/__init__.py", line 40, in detect
        if spec.can_handle():
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/specs/yaml_file.py", line 18, in can_handle
        self._environment = env.from_file(self.filename)
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/env.py", line 144, in from_file
        return from_yaml(yamlstr, filename=filename)
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/env.py", line 129, in from_yaml
        data = yaml_load_standard(yamlstr)
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda/common/serialize.py", line 76, in yaml_load_standard
        return yaml.load(string, Loader=yaml.Loader, version="1.2")
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/main.py", line 640, in load
        return loader._constructor.get_single_data()  # type: ignore
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/constructor.py", line 102, in get_single_data
        node = self.composer.get_single_node()
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 75, in get_single_node
        document = self.compose_document()
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 96, in compose_document
        node = self.compose_node(None, None)
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 132, in compose_node
        node = self.compose_mapping_node(anchor)
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 187, in compose_mapping_node
        while not self.parser.check_event(MappingEndEvent):
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/parser.py", line 144, in check_event
        self.current_event = self.state()
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/parser.py", line 547, in parse_block_mapping_key
        if self.scanner.check_token(KeyToken):
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/scanner.py", line 160, in check_token
        self.fetch_more_tokens()
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/scanner.py", line 273, in fetch_more_tokens
        return self.fetch_value()
      File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/scanner.py", line 626, in fetch_value
        self.reader.get_mark())
    ruamel_yaml.scanner.ScannerError: mapping values are not allowed here
      in "<unicode string>", line 2, column 17:
            dependencies:
                        ^ (line: 2)

`$ /Users/macbookair13/anaconda3/bin/conda-env create -f hello-spark.yml`

  environment variables:
                 CIO_TEST=<not set>
  CONDA_AUTO_UPDATE_CONDA=false
        CONDA_BACKUP_HOST=x86_64-apple-darwin13.4.0
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=/Users/macbookair13/anaconda3/bin/conda
             CONDA_PREFIX=/Users/macbookair13/anaconda3
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=/Users/macbookair13/anaconda3/bin/python
               CONDA_ROOT=/Users/macbookair13/anaconda3
              CONDA_SHLVL=1
                     PATH=/Users/macbookair13/anaconda3/bin:/Users/macbookair13/anaconda3/bin:/U
                          sers/macbookair13/anaconda3/condabin:/opt/local/bin:/opt/local/sbin:/u
                          sr/local/bin:/Users/macbookair13/anaconda/bin:/usr/local/bin:/usr/bin:
                          /bin:/usr/sbin:/sbin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : base
    active env location : /Users/macbookair13/anaconda3
            shell level : 1
       user config file : /Users/macbookair13/.condarc
 populated config files : /Users/macbookair13/.condarc
          conda version : 4.7.5
    conda-build version : 3.17.8
         python version : 3.7.3.final.0
       virtual packages : 
       base environment : /Users/macbookair13/anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/macbookair13/anaconda3/pkgs
                          /Users/macbookair13/.conda/pkgs
       envs directories : /Users/macbookair13/anaconda3/envs
                          /Users/macbookair13/.conda/envs
               platform : osx-64
             user-agent : conda/4.7.5 requests/2.21.0 CPython/3.7.3 Darwin/16.7.0 OSX/10.12.6
                UID:GID : 501:20
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.

If submitted, this report will be used by core maintainers to improve
future releases of conda.
Would you like conda to send this report to the core maintainers?

[y/N]: 

1 个答案:

答案 0 :(得分:1)

原始帖子按如下方式创建.yml文件:

name: hello-spark 
    dependencies:

    - python=3.6
    - jupyter
    - ipython
    - numpy
    - numpy-base
    - pandas
    - py4j
    - pyspark
    - pytz

正确的方法是:

name: hello-spark 
dependencies:
    - python=3.6
    - jupyter
    - ipython
    - numpy
    - numpy-base
    - pandas
    - py4j
    - pyspark
    - pytz

如果不确定如何在vi中进行编辑,只需使用sublime文本或任何其他代码编辑器即可。