Question

我正在处理this page上找到的hadoop示例，并且遇到了找不到Class的错误。 Eclipse没有看到任何语法错误，甚至在我在job.setMapperClass(TokenizerMapper.class)中突出显示实例时突出显示了TokenizerMapper类。这是因为它是一个子类，还是我在这里忽略了什么？我使用命令hadoop jar word.jar input output从主Hadoop节点执行它（我已经发现目录（args）参数是HDFS中/ user / [myuser]的相对路径，因此＆＃39;不是问题。

任何帮助都将不胜感激。

    import java.io.IOException;
    import java.util.StringTokenizer;
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.IntWritable;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.mapreduce.Mapper;
    import org.apache.hadoop.mapreduce.Reducer;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

    public class WordCount {

      Configuration configuration = null;

      public static class TokenizerMapper
        extends Mapper<Object, Text, Text, IntWritable>{

        private final static IntWritable one = new IntWritable(1);
        private Text word = new Text();

        public void map(Object key, Text value, Context context
                        ) throws IOException, InterruptedException {
          StringTokenizer itr = new StringTokenizer(value.toString());
          while (itr.hasMoreTokens()) {
            word.set(itr.nextToken());
            context.write(word, one);
          }
        }
      }

      public static class IntSumReducer
           extends Reducer<Text,IntWritable,Text,IntWritable> {
        private IntWritable result = new IntWritable();

        public void reduce(Text key, Iterable<IntWritable> values,
                           Context context
                           ) throws IOException, InterruptedException {
          int sum = 0;
          for (IntWritable val : values) {
            sum += val.get();
          }
          result.set(sum);
          context.write(key, result);
        }
      }

      public static void main(String[] args) {
        Configuration conf = new Configuration();
        Job job;
        conf.addResource(new Path("/work/hadoop/config","core-site.xml"));
    conf.addResource(new Path("/work/hadoop/config","hdfs-site.xml"));

    try {
        job = Job.getInstance(conf, "word count");


        job.setJarByClass(WordCount.class);
        job.setMapperClass(TokenizerMapper.class);
        job.setCombinerClass(IntSumReducer.class);
        job.setReducerClass(IntSumReducer.class);

        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);

        FileInputFormat.addInputPath(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job, new Path(args[1]));

        System.exit(job.waitForCompletion(true) ? 0 : 1);

        } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (ClassNotFoundException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (InterruptedException e) {
        // TODO Auto-generated catch block
            e.printStackTrace();
        }
      }
    }

Answer 1

您缺少命令行中的jar名称。在您的文档中搜索此内容。除了wc.jar之外，它还有WordCount，它是你的Driver类名。

bin/hadoop jar wc.jar WordCount /user/joe/wordcount/input /user/joe/wordcount/output

Answer 2

当您的jar应用程序入口点是您的WordCount.java时，您可以运行此命令而不会出现任何问题。 hadoop jar word.jar input output。目前jar并没有切入点。因此，如果您未在jar文件中指定Class的完全限定名称，则会出错通过使用Eclipse，您可以将类作为默认类/入口点运行，如下所示。

Project=> Right click => Export => JAR File => Next => Specify Jar Path => Next 
=> JAR Manifest specification.
select the class of the application entry point.  
Main class. Browse and select the main class(WordCount in your case).
Finish.

希望它有所帮助！

Answer 3

将JAR文件导出到类路径定义的目录可以解决问题。从eclipse导出JAR文件时，您可能会遇到“资源与文件系统不同步”错误。在这种情况下，只需右键单击项目并选择Refresh on eclipse。这将解决“不同步”问题

在Hadoop示例程序中找不到类WordCount $ TokenMapper

3 个答案: