H3作业未在s3n / impala上运行无法打开hdfs文件s3n:// my_bucket /

时间:2017-01-23 10:50:23

标签: amazon-s3 hive hdfs impala

我在fs.s3n.awsAccessKeyIdfs.s3n.awsSecretAccessKeycore中添加了hdfshive-site.xml个属性。我可以运行S3上的select * from tbl并使用beeline获得结果。但是,当我运行select count(*) from tbl时,它会失败:

Error: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
       ........
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:251)
        ... 11 more
Caused by: java.io.IOException: s3n://rakeshs3 : 400 : Bad Request
        at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:453)
        at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:427)
                 ... 16 more
Caused by: org.jets3t.service.impl.rest.HttpException: 400 Bad Request
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:425)
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279)
              ... 29 more

Impala错误:

Failed to open HDFS file s3n://rakeshs3/tel.txt    Error(255): Unknown error 255

0 个答案:

没有答案