Airflow S3转储问题

时间:2018-02-18 02:47:41

标签: python airflow

我有以下代码:

s3_hook = S3Hook(s3_conn_id=self.s3_conn_id)
a_key, s_key = s3_hook.get_credentials()
s3_hook.load_string(string_data=df_decrypted_string,
   bucket_name=self.s3_bucket_name,
   key=a_key,
   replace=False,
   encrypt=False)

但是,我得到以下问题

[2018-02-18 02:44:29,394] {base_task_runner.py:95} INFO - Subtask: 
[2018-02-18 02:44:29,390] {models.py:1433} ERROR - S3ResponseError: 404 Not Found
[2018-02-18 02:44:29,394] {base_task_runner.py:95} INFO - Subtask: Traceback (most recent call last):
[2018-02-18 02:44:29,394] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.6/site-packages/airflow/models.py", line 1390, in run
[2018-02-18 02:44:29,394] {base_task_runner.py:95} INFO - Subtask:     result = task_copy.execute(context=context)
[2018-02-18 02:44:29,394] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/airflow/plugins/decrypt.py", line 112, in execute
[2018-02-18 02:44:29,394] {base_task_runner.py:95} INFO - Subtask:     encrypt=False)
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py", line 414, in load_string
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:     bucket = self.get_bucket(bucket_name)
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py", line 222, in get_bucket
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:     return self.connection.get_bucket(bucket_name)
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.6/site-packages/boto/s3/connection.py", line 509, in get_bucket
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:     return self.head_bucket(bucket_name, headers=headers)
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.6/site-packages/boto/s3/connection.py", line 553, in head_bucket
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask:     raise err
[2018-02-18 02:44:29,395] {base_task_runner.py:95} INFO - Subtask: boto.exception.S3ResponseError: S3ResponseError: 404 Not Found

我试图通过其他人阅读'代码和Airflow指南无济于事。谁知道出了什么问题?

0 个答案:

没有答案