Question

我想将计数平均分数hadoop程序改为可以输出所有分数以及平均分数和总分的程序。在我修改程序之前，它可以在Eclipse中正确输出平均分数。但是在我修改它之后，它没有得到输出。因为我是Hadoop和Java的新手，所以我将我的程序与许多可行的程序进行了比较，但我无法找出发生的事情。

可行的减少：

 public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable>{
     public void reduce(Text key, Iterable<IntWritable> values, Context context)
        throws IOException, InterruptedException{
        int sum = 0;
        int count = 0;
        Iterator<IntWritable> iterator = values.iterator();
        while (iterator.hasNext()){
            sum += iterator.next().get();
            count++;
        }
        int average = (int) sum / count;
        context.write(key, new IntWritable(average));
    }
}

修改后的Reduce：

public static class Reduce extends Reducer<Text, IntWritable, Text, Text>{
    public void reduce(Text key, Iterable<IntWritable> values, Context context)
            throws IOException, InterruptedException{
        int sum = 0;
        int count = 0;
        String scoreList = new String();
        Iterator<IntWritable> iterator = values.iterator();
        while (iterator.hasNext()){
            int score = iterator.next().get();
            sum += score;
            count++;
            scoreList += String.format(" %d", score);
        }
        int average = (int) sum / count;
        scoreList += String.format(" %d", average);
        scoreList += String.format(" %d", sum);
        context.write(key, new Text(scoreList));
    }
}

修改后的主要内容：

public static void main(String[] args) throws Exception{
    Configuration conf = new Configuration();
    conf.set("fs.default.name", "hdfs://localhost:9000");
    String[] ioArgs = new String[] {"score_in", "score_out2"};
    String[] otherArgs = new GenericOptionsParser(conf, ioArgs).getRemainingArgs();
    if (otherArgs.length != 2) {
        System.err.println("Usage: Score <input> <output>");
        System.exit(2);
    }

    String inputDirName = otherArgs[0];
    String outputDirName = otherArgs[1];
    Job job = new Job(conf, "Score2");
    job.setJarByClass(Score2.class);
    job.setMapperClass(Map.class);
    job.setCombinerClass(Reduce.class);
    job.setReducerClass(Reduce.class);
    job.setMapOutputKeyClass(Text.class);
    job.setMapOutputValueClass(IntWritable.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(Text.class);
    job.setInputFormatClass(TextInputFormat.class);
    job.setOutputFormatClass(TextOutputFormat.class);
    FileInputFormat.addInputPath(job, new Path(inputDirName));
    FileOutputFormat.setOutputPath(job, new Path(outputDirName));
    System.exit(job.waitForCompletion(true) ? 0 : 1);
}

有人可以帮我做我的程序吗？非常感谢你！

简单的java hadoop程序（计算平均分数）没有用

0 个答案: