我正在按照here教程在AWS上进行PySpark。
我的操作系统:macOS High Sierra 10.12.6
到目前为止,一切都按教程中的方式进行。
我已经成功创建了“ hello-spark.yml”文件,并以精美的文字打开了文件,编辑的部分也在那里。
我运行以下代码时收到错误消息:
conda env create -f hello-spark.yml
myComputerName anaconda3 MyPersonalName$ conda env create -f hello-spark.yml
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda/exceptions.py", line 1043, in __call__
return func(*args, **kwargs)
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/cli/main.py", line 73, in do_call
exit_code = getattr(module, func_name)(args, parser)
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/cli/main_create.py", line 77, in execute
directory=os.getcwd())
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/specs/__init__.py", line 40, in detect
if spec.can_handle():
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/specs/yaml_file.py", line 18, in can_handle
self._environment = env.from_file(self.filename)
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/env.py", line 144, in from_file
return from_yaml(yamlstr, filename=filename)
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda_env/env.py", line 129, in from_yaml
data = yaml_load_standard(yamlstr)
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/conda/common/serialize.py", line 76, in yaml_load_standard
return yaml.load(string, Loader=yaml.Loader, version="1.2")
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/main.py", line 640, in load
return loader._constructor.get_single_data() # type: ignore
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/constructor.py", line 102, in get_single_data
node = self.composer.get_single_node()
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 75, in get_single_node
document = self.compose_document()
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 96, in compose_document
node = self.compose_node(None, None)
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 132, in compose_node
node = self.compose_mapping_node(anchor)
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/composer.py", line 187, in compose_mapping_node
while not self.parser.check_event(MappingEndEvent):
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/parser.py", line 144, in check_event
self.current_event = self.state()
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/parser.py", line 547, in parse_block_mapping_key
if self.scanner.check_token(KeyToken):
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/scanner.py", line 160, in check_token
self.fetch_more_tokens()
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/scanner.py", line 273, in fetch_more_tokens
return self.fetch_value()
File "/Users/macbookair13/anaconda3/lib/python3.7/site-packages/ruamel_yaml/scanner.py", line 626, in fetch_value
self.reader.get_mark())
ruamel_yaml.scanner.ScannerError: mapping values are not allowed here
in "<unicode string>", line 2, column 17:
dependencies:
^ (line: 2)
`$ /Users/macbookair13/anaconda3/bin/conda-env create -f hello-spark.yml`
environment variables:
CIO_TEST=<not set>
CONDA_AUTO_UPDATE_CONDA=false
CONDA_BACKUP_HOST=x86_64-apple-darwin13.4.0
CONDA_DEFAULT_ENV=base
CONDA_EXE=/Users/macbookair13/anaconda3/bin/conda
CONDA_PREFIX=/Users/macbookair13/anaconda3
CONDA_PROMPT_MODIFIER=(base)
CONDA_PYTHON_EXE=/Users/macbookair13/anaconda3/bin/python
CONDA_ROOT=/Users/macbookair13/anaconda3
CONDA_SHLVL=1
PATH=/Users/macbookair13/anaconda3/bin:/Users/macbookair13/anaconda3/bin:/U
sers/macbookair13/anaconda3/condabin:/opt/local/bin:/opt/local/sbin:/u
sr/local/bin:/Users/macbookair13/anaconda/bin:/usr/local/bin:/usr/bin:
/bin:/usr/sbin:/sbin
REQUESTS_CA_BUNDLE=<not set>
SSL_CERT_FILE=<not set>
active environment : base
active env location : /Users/macbookair13/anaconda3
shell level : 1
user config file : /Users/macbookair13/.condarc
populated config files : /Users/macbookair13/.condarc
conda version : 4.7.5
conda-build version : 3.17.8
python version : 3.7.3.final.0
virtual packages :
base environment : /Users/macbookair13/anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /Users/macbookair13/anaconda3/pkgs
/Users/macbookair13/.conda/pkgs
envs directories : /Users/macbookair13/anaconda3/envs
/Users/macbookair13/.conda/envs
platform : osx-64
user-agent : conda/4.7.5 requests/2.21.0 CPython/3.7.3 Darwin/16.7.0 OSX/10.12.6
UID:GID : 501:20
netrc file : None
offline mode : False
An unexpected error has occurred. Conda has prepared the above report.
If submitted, this report will be used by core maintainers to improve
future releases of conda.
Would you like conda to send this report to the core maintainers?
[y/N]:
答案 0 :(得分:1)
原始帖子按如下方式创建.yml文件:
name: hello-spark
dependencies:
- python=3.6
- jupyter
- ipython
- numpy
- numpy-base
- pandas
- py4j
- pyspark
- pytz
正确的方法是:
name: hello-spark
dependencies:
- python=3.6
- jupyter
- ipython
- numpy
- numpy-base
- pandas
- py4j
- pyspark
- pytz
如果不确定如何在vi中进行编辑,只需使用sublime文本或任何其他代码编辑器即可。