Question

我向Hadoop提交了MapReduce作业，并在屏幕上观看了进度报告。对于地图任务和减少任务，进度报告应单调增加（例如0％，10％，25％，60％，78％，95％和100％）。但事实上，所报告的进展并非单调增加：

14/01/21 11:05:37 INFO mapred.JobClient: Running job: job_201401201555_0036
14/01/21 11:05:38 INFO mapred.JobClient:  map 0%  reduce 0%
14/01/21 11:06:07 INFO mapred.JobClient:  map 11% reduce 0%
14/01/21 11:06:10 INFO mapred.JobClient:  map 0%  reduce 0%
14/01/21 11:06:19 INFO mapred.JobClient:  map 9%  reduce 0%
14/01/21 11:06:22 INFO mapred.JobClient:  map 22% reduce 0%
14/01/21 11:06:25 INFO mapred.JobClient:  map 31% reduce 0%
14/01/21 11:06:28 INFO mapred.JobClient:  map 39% reduce 0%
14/01/21 11:06:29 INFO mapred.JobClient:  map 53% reduce 0%
14/01/21 11:06:30 INFO mapred.JobClient:  map 57% reduce 0%
14/01/21 11:06:32 INFO mapred.JobClient:  map 50% reduce 0%
14/01/21 11:06:33 INFO mapred.JobClient:  map 55% reduce 0%
14/01/21 11:06:34 INFO mapred.JobClient:  map 43% reduce 0%
14/01/21 11:06:35 INFO mapred.JobClient:  map 48% reduce 0%
14/01/21 11:06:36 INFO mapred.JobClient:  map 40% reduce 0%
14/01/21 11:06:38 INFO mapred.JobClient:  map 30% reduce 0%
14/01/21 11:06:40 INFO mapred.JobClient:  map 40% reduce 0%
14/01/21 11:06:41 INFO mapred.JobClient:  map 49% reduce 0%
14/01/21 11:06:43 INFO mapred.JobClient:  map 57% reduce 0%
14/01/21 11:06:44 INFO mapred.JobClient:  map 70% reduce 0%
14/01/21 11:06:46 INFO mapred.JobClient:  map 73% reduce 0%
14/01/21 11:06:47 INFO mapred.JobClient:  map 82% reduce 0%
14/01/21 11:06:48 INFO mapred.JobClient:  map 93% reduce 0%
14/01/21 11:06:50 INFO mapred.JobClient:  map 94% reduce 0%
14/01/21 11:06:52 INFO mapred.JobClient:  map 95% reduce 0%
14/01/21 11:06:53 INFO mapred.JobClient:  map 96% reduce 0%
14/01/21 11:06:56 INFO mapred.JobClient:  map 98% reduce 0%
14/01/21 11:06:59 INFO mapred.JobClient:  map 99% reduce 0%
14/01/21 11:07:00 INFO mapred.JobClient:  map 100% reduce 0%
14/01/21 11:07:19 INFO mapred.JobClient:  map 100% reduce 4%
14/01/21 11:07:22 INFO mapred.JobClient:  map 100% reduce 8%
14/01/21 11:07:25 INFO mapred.JobClient:  map 100% reduce 66%
14/01/21 11:07:29 INFO mapred.JobClient:  map 100% reduce 67%
14/01/21 11:07:32 INFO mapred.JobClient:  map 100% reduce 68%
14/01/21 11:07:35 INFO mapred.JobClient:  map 100% reduce 69%
14/01/21 11:07:41 INFO mapred.JobClient:  map 100% reduce 70%
14/01/21 11:07:47 INFO mapred.JobClient:  map 100% reduce 71%
14/01/21 11:07:53 INFO mapred.JobClient:  map 100% reduce 72%
14/01/21 11:07:59 INFO mapred.JobClient:  map 100% reduce 73%
14/01/21 11:08:02 INFO mapred.JobClient:  map 100% reduce 100%
14/01/21 11:08:03 INFO mapred.JobClient: Job complete: job_201401201555_0036

进度由已在所有输入拆分中处理的百分比拆分表示。但为什么进度报告不是单调增加的呢？

Answer 1

检查tasktrackers和jobtracker的日志。地图阶段是否有任何失败？如果机器无法执行任务，或者主机无法再到达任务，则再次从另一台机器开始执行任务;所以进步不是单调增加的。

为什么MapReduce进度报告不是单调增加的？

1 个答案: