Question

我正在运行ETL脚本，用于将表从RDS获取到s3，但是ETL脚本始终失败，当我查看错误日志时，它显示了

的错误

Traceback (most recent call last):
File "script_2019-05-20-04-16-28.py", line 40, in <module>
datasink4 = glueContext.write_dynamic_frame.from_options(frame = dropnullfields3, connection_type = "s3", connection_options =
{
    "path": "s3://sd-datalake/data/enriched_archive/enriched/rm/metrics_archive/"
}
, format = "parquet", transformation_ctx = "datasink4")
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/PyGlue.zip/awsglue/dynamicframe.py", line 585, in from_options
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/PyGlue.zip/awsglue/context.py", line 193, in write_dynamic_frame_from_options
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/PyGlue.zip/awsglue/context.py", line 216, in write_from_options
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/PyGlue.zip/awsglue/data_sink.py", line 32, in write
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/PyGlue.zip/awsglue/data_sink.py", line 28, in writeFrame
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/mnt/yarn/usercache/root/appcache/application_1558325358249_0001/container_1558325358249_0001_01_000001/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o126.pyWriteDynamicFrame.
: java.io.IOException: Failed to delete key: data/enriched_archive/enriched/rm/metrics_archive/_temporary
at com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem.delete(S3NativeFileSystem.java:689)
at com.amazon.ws.emr.hadoop.fs.EmrFileSystem.delete(EmrFileSystem.java:296)

我怀疑问题可能是因为这样我在雅典娜中没有表，但是我只想将一个RDS表存储在s3中，并且为此我正在运行ETL，但它始终显示上述错误。

AWS Glue作业，用于将数据从RDS获取到S3

0 个答案: