hadoop没有将输出写入文件

Question

我的MapReduce代码出现问题。我的代码将从输入到输出文件写入最高库存和相应的名称。问题是正在写入零字节，我在输出中得到一个空文件。

HighestStock.java

JobConf conf = new JobConf(HighestStock.class);
conf.setJobName("Highest Stock");

FileInputFormat.addInputPath(conf, new Path(args[0]);
FileOutputFormat.setOutputPath(conf, new Path(args[1]);

conf.setMapperClass(HighStockMapper.class);
conf.setReducerClass(HighStockReducer.class);

conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);

JobClient.runJob(conf);

HighStockMapper.java

 public class HighStockMapper implements Mapper<LongWritable, Text, Text, IntWritable> {
 public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
            String line = value.toString();
            String[] tokens = line.split(",");
            String name = tokens[0];
               int high = (int) Double.parseDouble(tokens[3]);
               context.write(new Text(name), new IntWritable(high);
             }

HighStockReducer.java

public class HighStockReducer extends MapReduceBase
implements Reducer<Text, IntWritable, Text, IntWritable> {
      public void reduce (Tex key, Iterator<IntWritable> values,
            OutputCollector<Text, IntWritable> output, Reporter reporter)
      throws IOExceptiion {
            int maxValue = Integer.MIN_VALUE;
            while (values.hasNext()) {
                  maxValue = Math.max(maxValue, values.next().get());
            }
            output.collect(key, new IntWritable(maxValue));
       }
  }

任何有关调试的帮助都将非常感谢！

命令行结果的屏幕截图

Answer 1

你确实有一些拼写错误。我不认为这甚至可以编译..

context.write(new Text(name), new IntWritable(high); // Missing closing )

和

throws IOExceptiion { // extra i

我认为这不是您正在运行的实际代码？你可以发布运行作业的命令行结果（包括命令和计数器输出）吗？

编辑：@ anna-mai，你的截图显示映射器没有发出任何记录（Map输出记录= 0），因此问题出现在你的映射器中。

我注意到你遗失了mapper中的MapReduceBase。尝试添加：

public class HighStockMapper extends MapReduceBase implements Mapper

Answer 2

如前所述，您正在混合使用较旧和较新的实现。您需要对映射器类进行以下更改

class HighStockMapper extends MapReduceBase  implements Mapper<LongWritable, Text, Text, IntWritable> {

public void map(LongWritable key, Text value,
        OutputCollector<Text, IntWritable> output, Reporter arg3)
        throws IOException {
    String line = value.toString();
     String[] tokens = line.split(",");
     String name = tokens[0];
        int high = (int) Double.parseDouble(tokens[3]);
        output.collect(new Text(name), new IntWritable(high));

}

并且代码运行正常。