Question

我正在关注这个非常简单的教程（http://www.tutorialspoint.com/hadoop/hadoop_mapreduce.htm），一切运行正常，直到我尝试运行java文件。

我得到以下堆栈跟踪。

Exception in thread "main" java.lang.ClassNotFoundException: ProcessUnits
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

我尝试了很多不同的方法，但我总是得到ClassNotFoundException。我觉得它与Hadoop类路径或其他东西有关。请帮忙。

Answer 1

import java.io.IOException;
import java.util.Iterator;
import java.util.StringTokenizer;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reducer;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapred.TextInputFormat;
import org.apache.hadoop.mapred.TextOutputFormat;

public class ProcessUnits 
{ 
   //Mapper class 
   public static class E_EMapper extends MapReduceBase implements 
   Mapper<LongWritable ,/*Input key Type */ 
   Text,                /*Input value Type*/ 
   Text,                /*Output key Type*/ 
   IntWritable>        /*Output value Type*/ 
   { 

      //Map function 
      public void map(LongWritable key, Text value, 
      OutputCollector<Text, IntWritable> output,   
      Reporter reporter) throws IOException 
      { 
         String line = value.toString(); 
         String lasttoken = null; 
         StringTokenizer s = new StringTokenizer(line,"\t"); 
         String year = s.nextToken(); 

         while(s.hasMoreTokens())
            {
               lasttoken=s.nextToken();
            } 
         int avgprice = Integer.parseInt(lasttoken.trim()); 
         output.collect(new Text(year), new IntWritable(avgprice)); 
      } 
   } 


   //Reducer class 
   public static class E_EReduce extends MapReduceBase implements 
   Reducer< Text, IntWritable, Text, IntWritable > 
   {  

      //Reduce function 
      public void reduce( Text key, Iterator <IntWritable> values, 
         OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException 
         { 
            int maxavg=30; 
            int val=Integer.MIN_VALUE; 

            while (values.hasNext()) 
            { 
               if((val=values.next().get())>maxavg) 
               { 
                  output.collect(key, new IntWritable(val)); 
               } 
            } 

         } 
   }  
   //Main function 
   public static void main(String args[])throws Exception 
   { 
      JobConf conf = new JobConf(ProcessUnits.class); 
      conf.setJobName("max_eletricityunits"); 
      conf.setOutputKeyClass(Text.class);
      conf.setOutputValueClass(IntWritable.class); 
      conf.setMapperClass(E_EMapper.class); 
      conf.setCombinerClass(E_EReduce.class); 
      conf.setReducerClass(E_EReduce.class); 
      conf.setInputFormat(TextInputFormat.class); 
      conf.setOutputFormat(TextOutputFormat.class); 

      FileInputFormat.setInputPaths(conf, new Path(args[0])); 
      FileOutputFormat.setOutputPath(conf, new Path(args[1])); 

      JobClient.runJob(conf); 
   } 
}

此代码工作正常。我认为ProcessUnits是主要类，你已经更改了主类的名称。 JobConf conf = new JobConf(ProcessUnits.class);根据您的主要班级，您没有改变。根据你的课程主要课程给班级主要工作。

Answer 2

尝试在HADOOP_CLASSPATH中添加您的jar文件，并验证您是否编写了类mypackage.MyClass的完整规范名称。

Answer 3

你能检查一下你是否正在运行命令（检查当前目录和类名，你应该有hadoop.ProcessUnits而不仅仅是ProcessUnits，见下文）。

$ HADOOP_HOME / bin / hadoop jar units.jar hadoop.ProcessUnits input_dir output_dir

如果仍然无效，请尝试以下命令

$ HADOOP_HOME / bin / hadoop jar units.jar -cp ./* hadoop.ProcessUnits input_dir output_dir

Answer 4

如果在构建jar文件的过程中出现错误，也会发生这种情况，该文件将被用作＆＃34; hadoop jar＆＃34;的参数。这可能导致jar文件清单中的Main-class为＆＃34; null＆＃34;而不是null。这将阻止hadoop jar命令从命令行参数获取主类。

Hadoop：ClassNotFoundException

4 个答案: