Question

我正在使用带有MRv1的CDH4。据我所知，没有用于检查已完成作业的“状态”的命令行工具。当我转到Web控制台作业详细信息页面时，我可以看到“状态：失败”或“状态：成功”。如果我运行mapred job -list all或mapred job -status job_201309231203_0011，则均不表示“失败”或“成功”。

我错过了其他一些命令吗？

Answer 1

来自hadoop job -list all的第一对输出行是：

X jobs submitted
States are:
        Running : 1     Succeded : 2    Failed : 3      Prep : 4
JobId   State   StartTime       UserName        Priority        SchedulingInfo

输出线看起来像：

job_201309171413_38136  1       1382455374980   somebody        NORMAL  0 running map tasks using 0 map slots. 0 additional slots reserved. 1 running reduce tasks using 1 reduce slots. 0 additional slots reserved.
job_201309171413_37222  2       1382430339635   somebody        NORMAL  0 running map tasks using 0 map slots. 0 additional slots reserved. 0 running reduce tasks using 0 reduce slots. 0 additional slots reserved.

第二列是作业的State。根据标题行，1表示Running，2表示Succeeded。它不是最清晰的格式：4行标题，需要引用标题来确定状态代码的实际含义，并且无法获得一个作业的状态。

为特定作业解析此输出的最简单方法是：

$ job_id=job_201309171413_38136
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Running

$ job_id=job_201309171413_37222
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Succeeded

$ job_id=foobar
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Unknown

Answer 2

我的hadoop版本是2.5.0。这对我有用首先使用

获取job_id

hadoop job -list

然后通过获取job_id

来做

hadoop job  -status {job_id}

Answer 3

hadoop job -list all
hadoop job -status <JobID>

或 hadoop jobtracker Web仪表板将帮助您找到此错误或与工作相关的信息。

如何从命令行获取Hadoop作业的成功/失败状态？

3 个答案: