Question

你好，我正在通过aws Batch启动我的docker容器。我的aws批次不断失败。我目前正在尝试下载file_object并将其重新上传到其他s3存储桶。每次我收到OSERROR 第一次是：

OSError: [Errno 30] Read-only file system

这是我的下载功能：

 def download(self):
        s3 = boto3.client('s3')
        file_name = self.flow_cells[10:]
        try:
            with open(file_name, 'wb') as data:
                s3.download_fileobj(
                    self.source_s3_bucket,
                    self.source_key,
                    data
                )
            return True
        except botocore.exceptions.ClientError as error:
            print(error.response['Error']['Code'])

该错误发生在s3.download）fileobj调用中

它在命中数据时会被标记。

我第二次运行它来检查我得到的错误

OSError: [Errno 5] Input/output error

以下是我的容器定义。

container_properties = <<CONTAINER_PROPERTIES
  {
      "command": [
      "--object_key", "Ref::object_key",
      "--glacier_s3_bucket", "Ref::glacier_s3_bucket",
      "--output_s3_bucket", "Ref::output_s3_bucket",
      "--default_s3_bucket", "Ref::default_s3_bucket"  
      ],
      "environment": [],
      "image": "temp_image_name",
      "jobRoleArn": "${aws_iam_role.task-role.arn}",
      "memory": 1024,
      "mountPoints": [],
      "privileged": true,
      "readonlyRootFilesystem": false,
      "ulimits": [],
      "vcpus": 1,
      "volumes": [],
    "jobDefinitionName": "docker-flowcell-restore-${var.environment}"
  }

这是程序的完整日志：

File "src/main.py", line 101, in download
17:10:55
data
17:10:55
File "/usr/local/lib/python3.5/dist-packages/boto3/s3/inject.py", line 678, in download_fileobj
17:10:55
return future.result()
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/futures.py", line 73, in result
17:10:55
return self._coordinator.result()
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/futures.py", line 233, in result
17:10:55
raise self._exception
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/tasks.py", line 126, in __call__
17:10:55
return self._execute_main(kwargs)
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/tasks.py", line 150, in _execute_main
17:10:55
return_value = self._main(**kwargs)
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/download.py", line 583, in _main
17:10:55
fileobj.write(data)
17:10:55
OSError: [Errno 5] Input/output error

Answer 1

解决方案是

os.chdir('/tmp')

在将要运行的Docker容器的代码中。

AWS Batch docker s3.download_fileobj（）错误

1 个答案: