HBase ExportSnapshot无法与AWS临时凭证一起使用

时间:2019-05-15 04:30:08

标签: amazon-web-services hadoop amazon-s3 hbase aws-sdk

我正在尝试将hbase表快照从本地(linux)框中导出到s3存储桶。

使用以下命令:

HADOOP_CLASSPATH=~/hadoop-aws-2.6.0-cdh5.4.0.jar:~/aws-java-sdk-sts-1.11.550.jar ~/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -Dhadoop.tmp.dir=file:///home/users/aws/ -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec -Dfs.s3a.access.key=ASIAXXXXXX -Dfs.s3a.secret.key=XXXXXX -Dfs.s3a.session.token=XXXXX -Dfs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider -snapshot TABLE_NAME -copy-to s3a://bucket-name/

日志

2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Error Message: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: XXXXXX, AWS Error Code: null, AWS Error Message: Forbidden
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: HTTP Status Code: 403
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: AWS Error Code: null
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Error Type: Client
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Request ID: XXXXX
2019-05-14 06:22:59,532 INFO  [main] s3a.S3AFileSystem: Class Name: com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception
Exception in thread "main" com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: XXXXX, AWS Error Code: null, AWS Error Message: Forbidden, S3 Extended Request ID: XXXXX
  at com.cloudera.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
  at com.cloudera.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
  at com.cloudera.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
  at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
  at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:976)
  at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:956)
  at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:849)
  at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:76)
  at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
  at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:895)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:1024)
  at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:1028)```      

cli命令运行正常。
例如,

aws s3 ls
2019-01-10 23:19:47 zzzz
2019-05-06 13:31:21 xxxx

是否无法使用临时AWS凭证导出快照。

0 个答案:

没有答案