Question

我已经编写了一个mapreduce作业来进行日志文件分析。我的mappers输出文本作为键和值，我已经在我的驱动程序类中显式设置了地图输出类。

但是我仍然得到错误： - 来自map的键中的类型不匹配：期望org.apache.hadoop.io.Text，收到的org.apache.hadoop.io.LongWritable

public class CompositeUserMapper extends Mapper<LongWritable, Text, Text, Text> {

IntWritable a = new IntWritable(1);
//Text txt = new Text();

@Override
protected void map(LongWritable key, Text value,
        Context context)
        throws IOException, InterruptedException {
    String line = value.toString();

    Pattern p = Pattern.compile("\bd{8}\b");
    Matcher m = p.matcher(line);
    String userId = "";
    String CompositeId = "";
    if(m.find()){

        userId = m.group(1);
    }

     CompositeId = line.substring(line.indexOf("compositeId :")+13).trim();

     context.write(new Text(CompositeId),new Text(userId));


    // TODO Auto-generated method stub
    super.map(key, value, context);
}

我的驱动程序类如下： -

public class CompositeUserDriver extends Configured implements Tool {

public static void main(String[] args) throws Exception {

    CompositeUserDriver wd = new CompositeUserDriver();
    int res = ToolRunner.run(wd, args);
    System.exit(res);

}

public int run(String[] arg0) throws Exception {
    // TODO Auto-generated method stub

    Job job=new Job();
    job.setJarByClass(CompositeUserDriver.class);
    job.setJobName("Composite UserId Count" );

    FileInputFormat.addInputPath(job, new Path(arg0[0]));
    FileOutputFormat.setOutputPath(job, new Path(arg0[1]));
    job.setMapperClass(CompositeUserMapper.class);
    job.setReducerClass(CompositeUserReducer.class);
    job.setMapOutputKeyClass(Text.class);
    job.setMapOutputValueClass(Text.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);
    return job.waitForCompletion(true) ? 0 : 1;
    //return 0;
}

}

请告知如何解决这个问题。

Answer 1

从映射器代码中删除super.map(key, value, context);行：它调用父类的map方法，它是返回键和传递给它的值的标识映射器，在这种情况下，键是从头开始的字节偏移量文件

获取错误： - 键入map中的键不匹配：期望org.apache.hadoop.io.Text，收到org.apache.hadoop.io.LongWritable

1 个答案: