public class WcDriver {
public static void main(String[] args) throws IOException,
InterruptedException, ClassNotFoundException {
Configuration conf = new Configuration();
Job job = new Job(conf, "WcDriver");
job.setJarByClass(WcDriver.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(LongWritable.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
FileInputFormat.setInputPaths(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
job.setMapperClass(WcMapper.class);
job.setReducerClass(WcReducer.class);
job.waitForCompletion(true);
}
}
public class WcReducer extends Reducer<Text, LongWritable, Text,String>
{
@Override
public void reduce(Text key, Iterable<LongWritable> values, Context context) throws IOException, InterruptedException {
String key1 = null;
int total = 0;
for (LongWritable value : values) {
total += value.get();
key1= key.toString();
}
context.write(new Text(key1), "ABC");
}
}
在这里,在驱动程序类中我设置了job.setOutputKeyClass(Text.class)
和job.setOutputValueClass(LongWritable.class)
,但在reducer类中我正在编写一个字符串context.write(new Text(key1), "ABC");
。我认为运行程序时应该出错,因为输出类型不匹配,而reducer的密钥也应该实现WritableComparable
,值应该实现Writable
接口。奇怪的是,这个程序运行良好。我不明白为什么没有例外。
答案 0 :(得分:3)
尝试这样做:
// job.setOutputFormatClass(TextOutputFormat.class);
// comment this line, and you'll sure get exception of casting.
这是因为,TextOutputFormat假定LongWritable为键,Text为值,如果你不定义outPutFormat类,它会期望获得writable的默认行为,这是默认情况下,但如果你提到它,它会隐式地将它强制转换为给定的类型。;
答案 1 :(得分:1)
试试这个
//job.setOutputValueClass(LongWritable.class); if you comment this line you get an error
this will for only define the key value pair by defaul it depent on the output format and
it will be text so this is not giving any error