如何在减少地图处理中为每个工作人员收集地图或减少步骤结果?

时间:2018-10-29 19:04:09

标签: mapreduce

我想验证地图并减少步骤结果,所以我需要知道每个工作人员的结果。但我不知道如何收集每个工人(节点)的投入和产出。

  public static void main(String[] args) throws Exception {
    Configuration conf = new Configuration();
    Job job = Job.getInstance(conf, "word count");
    job.setJarByClass(WordCount.class);
    job.setMapperClass(TokenizerMapper.class);
    job.setCombinerClass(IntSumReducer.class);
    job.setReducerClass(IntSumReducer.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);
    FileInputFormat.addInputPath(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));
    System.exit(job.waitForCompletion(true) ? 0 : 1);
  }

0 个答案:

没有答案