我想知道是否还有检查由YARN启动的Spark应用程序读取或写入的HDFS字节数。例如,如果我检查YARN中完成的作业:
Total number of applications (application-types: [] and states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED]):2
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
application_1451137492021_0002 com.abrandon.upm.GenerateRandomText SPARK alvarobrandon default FINISHED SUCCEEDED 100% N/A
这个想法是能够监视application_1451137492021_0002读取或写入的字节数。我检查了datanode日志,但我只能找到非mapreduce作业的痕迹,当然也没有这个特定应用程序的痕迹。