Question

我正在开发 Hadoop 。我的输出是预期的两倍 我无法理解为什么会发生这种情况请帮帮我

以下是映射器类：

 import java.io.File;
 import java.io.IOException;
 import java.util.Scanner;
 import org.apache.hadoop.io.*;
 import org.apache.hadoop.mapred.*;

 public class StringMapper extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable>
 {
       //hadoop supported data types
       private static IntWritable send;
       private Text word;

       //map method that performs the tokenizer job and framing the initial key value  pairs
  public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException
  {
        String line = value.toString();
        String out="";
        int count=0;
            out+=Integer.toString(count);
            send = new IntWritable(1);
            word = new Text(out);
            output.collect(word, send);                                               
   }
 }

以下是 reducer类

import java.io.IOException;
import java.util.Iterator;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.*;

public class StringReducer extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable>
{
  //reduce method accepts the Key Value pairs from mappers, do the aggregation based on keys and produce the final output
  public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException
  {
        int sum=0;
        while(values.hasNext()){
            sum=sum+values.next().get();
        }
        output.collect(key, new IntWritable(sum));
  }
}

示例输入：
dashjdasdhashjfsda
dashjdasdhashjfsda
dashjdasdhashjfsda
dashjdasdhashjfsda
dashjdasdhashjfsda

示例输出
0 10

此处输出应为0 5而不是0 10，因为我的输入中只有五行。

Answer 1

您的程序似乎没问题。我复制了你的代码并在我的机器上运行它。它给出了正确的输出，即0 5

如果您正在使用eclipse，请创建一个新配置并更改您的输入目录。
然后它可能会工作。

Hadoop中Mapreduce程序中的意外输出

1 个答案: