使用HCATALOG

时间:2017-09-18 13:25:29

标签: sqoop

当我们调用Sqoop.runTool从Oracle数据库导入时。一旦Map And Reduce成为启动器,它就无法找到HCATALOG类:

  

2017-09-18 09:04:47,546 INFO [main] org.apache.hadoop.service.AbstractService:服务org.apache.hadoop.mapreduce.v2.app.MRAppMaster状态失败INITED;原因:org.apache.hadoop.yarn.exceptions.YarnRuntimeException:java.lang.RuntimeException:java.lang.ClassNotFoundException:未找到类org.apache.hive.hcatalog.mapreduce.HCatOutputFormat   org.apache.hadoop.yarn.exceptions.YarnRuntimeException:java.lang.RuntimeException:java.lang.ClassNotFoundException:未找到类org.apache.hive.hcatalog.mapreduce.HCatOutputFormat       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster $ 2.call(MRAppMaster.java:519)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster $ 2.call(MRAppMaster.java:499)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1598)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:499)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:285)       在org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster $ 5.run(MRAppMaster.java:1556)       at java.security.AccessController.doPrivileged(Native Method)       在javax.security.auth.Subject.doAs(Subject.java:422)       在org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)   引起:java.lang.RuntimeException:java.lang.ClassNotFoundException:未找到类org.apache.hive.hcatalog.mapreduce.HCatOutputFormat       在org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2214)       at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:222)       在org.apache.hadoop.mapreduce.v2.app.MRAppMaster $ 2.call(MRAppMaster.java:515)       ......还有11个   引起:java.lang.ClassNotFoundException:未找到类org.apache.hive.hcatalog.mapreduce.HCatOutputFormat       在org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2120)       在org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2212)       ......还有13个

知道如何解决这个问题吗?

1 个答案:

答案 0 :(得分:0)

最后,在调用Sqoop.runTool(s [],conf)hdalog路径中的hcatalog jar之前,我在配置对象中找到了解决方案设置

        conf.set("tmpjars", "/lib/hive-exec-1.2.1.2.3.4.0-3485.jar,/lib/hive-hcatalog-core-1.2.1.2.3.4.0-3485.jar,/lib/hive-jdbc-1.2.1.2.3.4.0-3485.jar...");