在运行某项工作时,我遇到以下异常:
java.io.FileNotFoundException: File file:/usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-*.jar does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:511)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
at org.apache.hadoop.mapreduce.JobSubmitter.copyRemoteFiles(JobSubmitter.java:140)
at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:213)
at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:72) Job Submission failed with exception 'java.io.FileNotFoundException(File file:/usr/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-*.jar does not exist)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
可能是什么原因,我该如何解决?丢失的jar文件确实存在:
[user@host ~]$ ls -l /usr/lib/hive-hcatalog/share/hcatalog/
total 664
-rw-r--r-- 1 root root 468839 May 27 12:59 hive-hcatalog-core-0.13.0.2.1.2.1-471.jar
lrwxrwxrwx 1 root root 41 Jun 30 15:57 hive-hcatalog-core.jar -> hive-hcatalog-core-0.13.0.2.1.2.1-471.jar
-rw-r--r-- 1 root root 81042 May 27 12:59 hive-hcatalog-pig-adapter-0.13.0.2.1.2.1-471.jar
lrwxrwxrwx 1 root root 48 Jun 30 15:57 hive-hcatalog-pig-adapter.jar -> hive-hcatalog-pig-adapter-0.13.0.2.1.2.1-471.jar
-rw-r--r-- 1 root root 67890 May 27 12:59 hive-hcatalog-server-extensions-0.13.0.2.1.2.1-471.jar
lrwxrwxrwx 1 root root 54 Jun 30 15:57 hive-hcatalog-server-extensions.jar -> hive-hcatalog-server-extensions-0.13.0.2.1.2.1-471.jar
-rw-r--r-- 1 root root 52552 May 27 12:59 hive-hcatalog-streaming-0.13.0.2.1.2.1-471.jar
lrwxrwxrwx 1 root root 46 Jun 30 15:57 hive-hcatalog-streaming.jar -> hive-hcatalog-streaming-0.13.0.2.1.2.1-471.jar
drwxr-xr-x 3 root root 4096 Jun 30 15:57 storage-handlers
我尝试使用此基准:https://github.com/cartershanklin/hive-testbench。运行tpcds-setup.sh
脚本时,恰好在以下行之后出现问题:
hive -i settings/load-partitioned.sql -f ddl-tpcds/bin_partitioned/store_sales.sql -d DB=tpcds_bin_partitioned_orc_6 -d SOURCE=tpcds_text_6 -d BUCKETS=1 -d RETURN_BUCKETS=1 -d FILE=orc
答案 0 :(得分:0)
https://issues.apache.org/jira/browse/HIVE-2379详述的错误提到了这种情况。如果不明确导出辅助jar,就无法测试Hive / Hcatalog集成。在export HIVE_AUX_JARS_PATH=/usr/lib/hcatalog/share/hcatalog/hcatalog-core.jar
中执行/etc/hive/conf/hive-env.sh
并添加hcatalog核心库的路径可以解决问题。
此致
阿迪尔