Question

我是cloudera经理新手，我使用cloudera manager v5.8安装CDH集群，我在集群中安装了hdfs，hbase，hive，MR2（yarn）等组件，用来运行一些程序，但是当我运行一个与mapreduce相关的程序时，我得到一个“ NoSuchMethodError：Org.apache.hadoop.mapreduce.Job.setJar“错误，我怀疑在安装CDH之后一些环境变量没有设置让局部mapreduce.client版本不正确，我尝试了各种方法设置环境变量可以没解决这个问题，我还检查了cloudera manager mapreduce相关的配置参数，但是仍然找不到相关设置来解决这个问题。

错误信息：

java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.Job.setJar(Ljava/lang/String;)V
at org.apache.kylin.engine.mr.common.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:163)
at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:88)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:88)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

环境：

    root@yinpiao2:~/apache-kylin-1.5.4-cdh5.7-bin/bin# cat /etc/profile
...
    export JAVA_HOME=/usr/java/jdk1.7.0_80
    export PATH=$PATH:$JAVA_HOME/bin
    export MAVEN_HOME=/data/maven
    export HADOOP_CLASSPATH=.:$HADOOP_CLASSPATH:/opt/cloudera-manager/cm-5.8.0/share/cmf/common_jars/hadoop-core-2.6.0-mr1-cdh5.5.0.jar
    export HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
    export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
    export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
    export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
    export ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/zookeeper
    export TOMCAT_HOME=/data/tomcat
    export CATALINA_HOME=/data/tomcat
    export CLASSPATH=.:$CLASSPATH:/usr/share/java/mysql-connector-java-5.1.28.jar:$JAVA_HOME/lib/tools.jar
    export KYLIN_HOME=/home/yinpiao/apache-kylin-1.5.4-cdh5.7-bin
    export hive_dependency=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/conf:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/lib/*:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/jars/hive-hcatalog-core-1.1.0-cdh5.8.0.jar
    export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$KYLIN_HOME/bin:$PATH:$PATH:$TOMCAT_HOME/bin

希望有人能回答这个问题，谢谢。

在Cloudera CDH 5.8中运行mapreduce时出现“NoSuchMethodError：org.apache.hadoop.mapreduce.Job.setJar”错误

0 个答案: