我是cloudera经理新手, 我使用cloudera manager v5.8安装CDH集群,我在集群中安装了hdfs,hbase,hive,MR2(yarn)等组件,用来运行一些程序,但是当我运行一个与mapreduce相关的程序时,我得到一个“ NoSuchMethodError:Org.apache.hadoop.mapreduce.Job.setJar“错误,我怀疑在安装CDH之后一些环境变量没有设置让局部mapreduce.client版本不正确,我尝试了各种方法设置环境变量可以没解决这个问题,我还检查了cloudera manager mapreduce相关的配置参数,但是仍然找不到相关设置来解决这个问题。
错误信息:
java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.Job.setJar(Ljava/lang/String;)V
at org.apache.kylin.engine.mr.common.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:163)
at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:88)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:88)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
环境:
root@yinpiao2:~/apache-kylin-1.5.4-cdh5.7-bin/bin# cat /etc/profile
...
export JAVA_HOME=/usr/java/jdk1.7.0_80
export PATH=$PATH:$JAVA_HOME/bin
export MAVEN_HOME=/data/maven
export HADOOP_CLASSPATH=.:$HADOOP_CLASSPATH:/opt/cloudera-manager/cm-5.8.0/share/cmf/common_jars/hadoop-core-2.6.0-mr1-cdh5.5.0.jar
export HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
export ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/zookeeper
export TOMCAT_HOME=/data/tomcat
export CATALINA_HOME=/data/tomcat
export CLASSPATH=.:$CLASSPATH:/usr/share/java/mysql-connector-java-5.1.28.jar:$JAVA_HOME/lib/tools.jar
export KYLIN_HOME=/home/yinpiao/apache-kylin-1.5.4-cdh5.7-bin
export hive_dependency=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/conf:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/lib/*:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/jars/hive-hcatalog-core-1.1.0-cdh5.8.0.jar
export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$KYLIN_HOME/bin:$PATH:$PATH:$TOMCAT_HOME/bin
希望有人能回答这个问题,谢谢。