Hadoop S3列表存储桶内容错误

时间:2015-11-29 17:48:24

标签: hadoop amazon-web-services amazon-s3

不确定我在这里缺少什么。解决使用hadoop 2.7.1从s3尝试存储桶内容时得到的以下消息的最佳方法是什么?这应该是非常直接的,我有我的core-site.xml和hdfs-site.xml文件,如下所示,然后我尝试运行hadoop fs -ls s3a://<bucket_name>

HDFS-site.xml中:

<configuration>
  <property>
    <name>fs.s3a.access.key</name>
    <value>key</value>
  </property>
  <property>
    <name>fs.s3a.secret.key</name>
    <value>secret</value>
  </property>
</configuration>

核心-site.xml中:

<configuration>
    <property>
        <name>fs.s3a.impl</name>
        <value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
    </property>
</configuration>

错误讯息:

[root@ip-10-239-197-136 ~]# hadoop fs -ls s3a://<bucket_name>
15/11/29 17:43:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-ls: Fatal internal error
com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: ED84F95A33096A67, AWS Error Code: null, AWS Error Message: Forbidden, S3 Extended Request ID: l4FDE3LnYtOSj0TNUrwqv3yX/3x3RgesasBWDo7WcdrS3rkn/6TCz9+Rt6uylVxGcMaztu7gYH8=
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
    at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:976)
    at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:956)
    at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:892)
    at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
    at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:57)
    at org.apache.hadoop.fs.Globber.glob(Globber.java:252)
    at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1655)
    at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:326)
    at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:235)
    at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:218)
    at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:201)
    at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
    at org.apache.hadoop.fs.FsShell.run(FsShell.java:287)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)

0 个答案:

没有答案