Question

我有一个方案来计算map reduce中两列的平均值。所以我做的是，我使用mapper从文件中获取值并将它们连接为Text然后尝试将它们写入Context，如下所示。

＆＃13;

class TestMapper extends Mapper<LongWritable, Text, Text, Text> {
  private Text outputKey;
  private Text outputVal;


  @Override
  public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
  
  //more code here

context.write(outputkey,OutputVal);


    
  }
}

＆＃13;

Answer 1

您应该在此处使用自定义数据类型，例如一个TextPair类，它有两个Text元素来存储您需要的数据。下面是一个示例代码，用于输出映射器上下文值的一对字符串。

// Mapper's map code
protected void map(LongWritable key, Text value, Mapper<LongWritable, Text, Text, TextPair>.Context context)
        throws IOException, InterruptedException {

    String line = value.toString();
    String year = line.substring(15, 19);
    int airTemperature;
    if (line.charAt(87) == '+') { // parseInt doesn't like leading plus signs
      airTemperature = Integer.parseInt(line.substring(88, 92));
    } else {
      airTemperature = Integer.parseInt(line.substring(87, 92));
    }
    String quality = line.substring(92, 93);
    if (airTemperature != MISSING && quality.matches("[01459]")) {
        System.out.println("Year "+year+" "+airTemperature);
      context.write(new Text(year), new TextPair(String.valueOf(airTemperature),1));
    }

//文本对 - 下面的自定义数据类型代码

public class TextPair implements WritableComparable<TextPair> {

private Text first;
private Text second;

//Default constructor is a must
public TextPair() {
    this.first=new Text();
    this.second=new Text();
}

public TextPair(String first,int second) {
    try {
        this.first=new Text(first);
        this.second=new Text(String.valueOf(second));
    }catch(Exception ex) {
        System.out.println("Exception occurred "+ex.getCause());
    }

}

// Other methods such as compare, equals, hashcode, write, readFields etc implementation also needs to done

public Text getFirst() {
    return first;
}

public Text getSecond() {
    return second;
}

@Override
public String toString() {
    return this.first+"\t"+this.second+"\t";
}

}

如果您还需要更多详细信息，请参阅Hadoop权威指南。希望这会有所帮助。

我可以使用Text作为值来写入mapreduce中的上下文

1 个答案: