我正在尝试通过运行
获取映射器中输入记录的行数job.getCounters().findCounter(TaskCounter.MAP_INPUT_RECORDS).getValue()
实际上,它在工作完成后可以工作,我想在减速器的设置阶段做同样的事情。我已尝试根据此页面Accessing a mapper's counter from a reducer覆盖2个设置功能。但是,它们都不起作用,并且都返回空指针异常。
@Override
protected void setup(Reducer<Text, DoubleWritable, Text, DoubleWritable>.Context context)
throws IOException, InterruptedException {
Configuration conf = context.getConfiguration();
Cluster cluster = new Cluster(conf);
//RunningJob job = (RunningJob) cluster.getJob(context.getJobID());
Job job = cluster.getJob(context.getJobID());
System.out.println(job.getCounters().findCounter(TaskCounter.MAP_INPUT_RECORDS).getValue());
}
@Override
protected void setup(Reducer<Text, DoubleWritable, Text, DoubleWritable>.Context context)
throws IOException, InterruptedException {
Configuration conf = context.getConfiguration();
JobClient client = new JobClient(conf);
RunningJob job = client.getJob(JobID.forName(conf.get("mapred.map.id")));
//Job job = (Job) client.getJob((JobID) context.getJobID());
System.out.println(job.getCounters().findCounter(TaskCounter.MAP_INPUT_RECORDS).getValue());
}
有人可以帮我解决问题吗?非常感谢。