我正在尝试将hbase表快照从本地(linux)框中导出到s3存储桶。
使用以下命令:
HADOOP_CLASSPATH=~/hadoop-aws-2.6.0-cdh5.4.0.jar:~/aws-java-sdk-sts-1.11.550.jar ~/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -Dhadoop.tmp.dir=file:///home/users/aws/ -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec -Dfs.s3a.access.key=ASIAXXXXXX -Dfs.s3a.secret.key=XXXXXX -Dfs.s3a.session.token=XXXXX -Dfs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider -snapshot TABLE_NAME -copy-to s3a://bucket-name/
日志
2019-05-14 06:22:59,532 INFO [main] s3a.S3AFileSystem: Error Message: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: XXXXXX, AWS Error Code: null, AWS Error Message: Forbidden
2019-05-14 06:22:59,532 INFO [main] s3a.S3AFileSystem: HTTP Status Code: 403
2019-05-14 06:22:59,532 INFO [main] s3a.S3AFileSystem: AWS Error Code: null
2019-05-14 06:22:59,532 INFO [main] s3a.S3AFileSystem: Error Type: Client
2019-05-14 06:22:59,532 INFO [main] s3a.S3AFileSystem: Request ID: XXXXX
2019-05-14 06:22:59,532 INFO [main] s3a.S3AFileSystem: Class Name: com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception
Exception in thread "main" com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: XXXXX, AWS Error Code: null, AWS Error Message: Forbidden, S3 Extended Request ID: XXXXX
at com.cloudera.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)
at com.cloudera.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)
at com.cloudera.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:976)
at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:956)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:849)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:76)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:895)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:1024)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:1028)```
cli命令运行正常。
例如,
aws s3 ls
2019-01-10 23:19:47 zzzz
2019-05-06 13:31:21 xxxx
是否无法使用临时AWS凭证导出快照。