Question

我正在尝试使用ml-engine jobs submit training启动cloud composer，我正在使用本指南来获取说明recommendation-system-tensorflow-deploy。

我使用的是Google创建的插件（see the implementation here）

我正在尝试使其在python 3.5版上运行，方法是将第206行更改为：

training_request = {
        'jobId': job_id,
        'trainingInput': {
            'scaleTier': self._scale_tier,
            'packageUris': self._package_uris,
            'pythonModule': self._training_python_module,
            'region': self._region,
            'args': self._training_args,
            'masterType': self._master_type
        }

收件人：

training_request = {
    'jobId': job_id,
    'trainingInput': {
        'scaleTier': self._scale_tier,
        'packageUris': self._package_uris,
        'pythonModule': self._training_python_module,
        'region': self._region,
        'args': self._training_args,
        'masterType': self._master_type,
        'python-version': '3.5'     #self._python_version

    }

我还尝试向其添加运行时版本（runtime-version='1.12'），但我不断遇到以下错误：

[2019-01-20 11:58:36,331] {models.py:1594} ERROR - <HttpError 400 when requesting https://ml.googleapis.com/v1/projects/hallowed-forge-577/jobs?alt=json returned "Invalid JSON payload received. Unknown name "python-version" at 'job.training_input': Cannot find field.">
Traceback (most recent call last)
  File "/usr/local/lib/airflow/airflow/models.py", line 1492, in _run_raw_tas
    result = task_copy.execute(context=context
  File "/home/airflow/gcs/plugins/ml_engine_plugin.py", line 241, in execut
    self._project_id, training_request, check_existing_job
  File "/home/airflow/gcs/plugins/ml_engine_plugin.py", line 79, in create_jo
    request.execute(
  File "/usr/local/lib/python3.6/site-packages/oauth2client/util.py", line 135, in positional_wrappe
    return wrapped(*args, **kwargs
  File "/usr/local/lib/python3.6/site-packages/googleapiclient/http.py", line 838, in execut
    raise HttpError(resp, content, uri=self.uri
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://ml.googleapis.com/v1/projects/hallowed-forge-577/jobs?alt=json returned "Invalid JSON payload received. Unknown name "python-version" at 'job.training_input': Cannot find field."
[2019-01-20 11:58:36,334] {models.py:1623} INFO - Marking task as FAILED.
[2019-01-20 11:58:36,513] {models.py:1627} ERROR - Failed to send email to: ['airflow@example.com']
[2019-01-20 11:58:36,516] {models.py:1628} ERROR - HTTP Error 401: Unauthorized
Traceback (most recent call last)
  File "/usr/local/lib/airflow/airflow/models.py", line 1625, in handle_failur
    self.email_alert(error, is_retry=False
  File "/usr/local/lib/airflow/airflow/models.py", line 1778, in email_aler
    send_email(task.email, title, body
  File "/usr/local/lib/airflow/airflow/utils/email.py", line 44, in send_emai
    return backend(to, subject, html_content, files=files, dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype
  File "/usr/local/lib/airflow/airflow/contrib/utils/sendgrid.py", line 116, in send_emai
    _post_sendgrid_mail(mail.get()
  File "/usr/local/lib/airflow/airflow/contrib/utils/sendgrid.py", line 122, in _post_sendgrid_mai
    response = sg.client.mail.send.post(request_body=mail_data
  File "/usr/local/lib/python3.6/site-packages/python_http_client/client.py", line 252, in http_reques
    return Response(self._make_request(opener, request, timeout=timeout)
  File "/usr/local/lib/python3.6/site-packages/python_http_client/client.py", line 176, in _make_reques
    raise ex
python_http_client.exceptions.UnauthorizedError: HTTP Error 401: Unauthorize

请注意，python版本实际上已更改（从原来的2.7更改为3.6），因此更改python版本会有所作为，但随后会卡住

任何对我在这里缺少的帮助都会很棒！

Answer 1

该示例似乎使用了旧版本的气流MLEngineTrainingOperator。最新版本实现了runtime-version / python-version训练参数。使用当前版本： mlengine_operator.py

ML引擎因作曲者而失败-未知名称“ python-version”

1 个答案: