Question

我在cloudera中运行地图减少工作

代码：

public class WordCount {
    public static class Map extends MapReduceBase implements Mapper<LongWritable,Text,Text,IntWritable>{
        private  final static IntWritable one = new IntWritable(1);
        private Text word = new Text();

        public void map(LongWritable key, Text value, OutputCollector<Text,IntWritable> output, Reporter reporter) throws IOException{
            String  line = value.toString();
            StringTokenizer tokenizer = new StringTokenizer(line);

            while(tokenizer.hasMoreTokens()){
                word.set(tokenizer.nextToken());
                output.collect(word, one);
            }
        }
    }

    public static class Reduce extends MapReduceBase implements Reducer<Text,IntWritable,Text,IntWritable>{

        public void reduce (Text key, Iterator<IntWritable> values,OutputCollector<Text,IntWritable> output,Reporter reporter)throws IOException {
            int sum = 0;
            while(values.hasNext()){
                sum += values.next().get();
            }
            output.collect(key  , new IntWritable(sum));

        }
    }

    public static void main(String[] args) throws Exception{
        JobConf conf = new JobConf(WordCount.class);
        conf.setJobName("WordCount");
        conf.setJarByClass(WordCount.class);
        conf.setOutputKeyClass(Text.class);
        conf.setOutputValueClass(IntWritable.class);

        conf.setMapperClass(Map.class);
        conf.setCombinerClass(Reduce.class);
        conf.setReducerClass(Reducer.class);


        conf.setInputFormat(TextInputFormat.class);
        conf.setOutputFormat(TextOutputFormat.class);

        FileInputFormat.setInputPaths(conf, new Path(args[0]));
        FileOutputFormat.setOutputPath(conf, new Path (args[1]) );

        JobClient.runJob(conf);


    }

}

命令：

[cloudera@localhost bin]$ hadoop jar /home/cloudera/workspace/Test/bin/WordCount.jar WordCount /user/cloudera/input /user/cloudera/output

错误：

14/02/28 22:49:34 INFO mapred.JobClient: Task Id : attempt_201402281818_0016_r_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.mapred.Reducer.<init>()
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:469)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.mapred.Reducer.<init>()
    at java.lang.Class.getConstructor0(Class.java:2706)
    at java.lang.Class.getDeclaredConstructor(Class.java:1985)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
    ... 7 more

请帮忙。

Answer 1

也许您应该尝试纠正这行代码：

conf.setReducerClass(Reducer.class);

到

conf.setReducerClass(Reduce.class);

因为我注意到你的reducer类的名字是Reduce。

地图减少工作失败

1 个答案: