Question

在Cloud Shell中运行this posting by Google's Slaven Bilac的示例代码时，会发生错误。

chuck@wordone-wordtwo-1234567:~/google-cloud-ml/samples/flowers$ ./sample.sh
Your active configuration is: [cloudshell-270]


Using job id:  flowers_chuck_20170130_212715

# Takes about 30 mins to preprocess everything.  We serialize the two
# preprocess.py synchronous calls just for shell scripting ease; you could use
# --runner DataflowPipelineRunner to run them asynchronously.  Typically,
# the total worker time is higher when running on Cloud instead of your local
# machine due to increased network traffic and the use of more cost efficient
# CPU's.  Check progress here: https://console.cloud.google.com/dataflow
python trainer/preprocess.py \
  --input_dict "$DICT_FILE" \
  --input_path "gs://cloud-ml-data/img/flower_photos/eval_set.csv" \
  --output_path "${GCS_PATH}/preproc/eval" \
  --cloud
WARNING:root:Couldn't find python-snappy so the implementation of _TFRecordUtil._masked_crc32c is not as fast as it could be.
WARNING:root:BlockingDataflowPipelineRunner is deprecated, use BlockingDataflowRunner instead.
WARNING:root:BlockingDataflowRunner is deprecated, use DataflowRunner instead.
Traceback (most recent call last):
  File "trainer/preprocess.py", line 446, in <module>
main(sys.argv[1:])

  File "trainer/preprocess.py", line 442, in main
run(arg_dict)
  File "trainer/preprocess.py", line 361, in run
p = beam.Pipeline(options=pipeline_options)
  File "/home/chuck/.local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 131, in __init__
    'Pipeline has validations errors: \n' + '\n'.join(errors))
ValueError: Pipeline has validations errors: 
Missing GCS path option: temp_location.

如何修改GoogleCloudPlatform/cloudml-samples/flowers中的文件以避免这种情况？

Answer 1

查克，

您看到的问题似乎是版本控制不匹配。简单的解决方法是使用preprocess.py中的default_values dic中的temp_location替换staging_location：

default_values = {
    'project':
        get_cloud_project(),
    # **here**  
    'temp_location':
        os.path.join(os.path.dirname(parsed_args.output_path), 'temp'),
    'runner':
        'DataflowRunner',
    'extra_package':
        Default.CML_PACKAGE,
    'save_main_session':
        True,
}

我们将很快更新样本，以反映这一点。

Answer 2

您是否尝试过运行CloudShell setup script？它有助于管理一些版本，这似乎是这里的核心问题：

curl https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/setup_cloud_shell.sh | bash

使用google cloud ml示例代码重新启动错误

2 个答案: