用于映射时间和减少时间的hadoop源代码中的命令

Question

我们知道map有两个部分'chunk and combine'，而reduce有3个部分'shuffle，sort and reduce'。 在hadoop源代码中，每个部分时间的命令是什么

Answer 1

JobTracker有一个API，用于在网络环境中提交和跟踪MR作业。 请查看此内容以获取更多详 https://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapred/JobTracker.html

    TaskReport[] maps = jobtracker.getMapTaskReports("job_id");
    for (TaskReport rpt : maps) {
      System.out.println(rpt.getStartTime());  
      System.out.println(rpt.getFinishTime());
    }
    TaskReport[] reduces = jobtracker.getReduceTaskReports("job_id");
    for (TaskReport rpt : reduces) {

      System.out.println(rpt.getStartTime());
System.out.println(rpt.getFinishTime());
    }

或者如果您使用的是Hadoop 2.x，则提供了ResourceManager REST API。

https://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html

https://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/HistoryServerRest.html