Question

我正在一段时间间隔内搜索所有Hadoop完成的（100个）作业。此时间间隔以毫秒为单位。

以下是格式：

JobId      State           StartTime      UserName           Queue      Priority       UsedContainers  RsvdContainers      UsedMem         RsvdMem        NeededMem                 AM info

job_xxxxxxx  SUCCEEDED       1458844667431    default    default          NORMAL                  N/A             N/A          N/A           N/A        N/A             http://xxxxxxxx:8088/proxy/application_xxxxxxxxxx/jobhistory/job/job_xxxxxxxx
job_xxxxxxx  SUCCEEDED       1459449718363    default    default         NORMAL                  N/A             N/A        N/A             N/A              N/A        http://xxxxxx.xxxxx.com:8088/proxy/application_xxxxxxxxx/jobhistory/job/job_xxxxx

以下是我的格式：

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"

CONVERTTIME_1=`date --date="$STARTTIME" +%s%3N`
CONVERTTIME_2=`date --date="$ENDTIME" +%s%3N`

echo $CONVERTTIME_1, $CONVERTTIME_2

mapred job -list all | sed -n '/$CONVERTTIME_1/,/$CONVERTTIME_2/p' > out

输出：timerange内的所有上述工作。

有人可以帮忙解决这些问题吗？

输出mapred job -list all

mapred job -list all

 job_1457613852865_5163  SUCCEEDED       1459199337140    zzzzzzzzzz    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5163/jobhistory/job/job_1457613852865_5163

 job_1457613852865_4633  SUCCEEDED       1458992402216       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4633/jobhistory/job/job_1457613852865_4633

 job_1457613852865_4821  SUCCEEDED       1459078845580       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4821/jobhistory/job/job_1457613852865_4821

 job_1457613852865_0322  SUCCEEDED       1457717313217        zzzddd    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_0322/jobhistory/job/job_1457613852865_0322

 job_1457613852865_5304  SUCCEEDED       1459254375921       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5304/jobhistory/job/job_1457613852865_5304

 job_1457613852865_8744  SUCCEEDED       1460195126188       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8744/jobhistory/job/job_1457613852865_8744

 job_1457613852865_3384  SUCCEEDED       1458649020794       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_3384/jobhistory/job/job_1457613852865_3384

 job_1457613852865_9038  SUCCEEDED       1460291694279       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_9038/jobhistory/job/job_1457613852865_9038

 job_1457613852865_8487  SUCCEEDED       1460115319590       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8487/jobhistory/job/job_1457613852865_8487

 job_1457613852865_8321  SUCCEEDED       1460038991587        dddyyy    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8321/jobhistory/job/job_1457613852865_8321

 job_1457613852865_4661  SUCCEEDED       1458994901619       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4661/jobhistory/job/job_1457613852865_4661

 job_1457613852865_1975  SUCCEEDED       1458216683800       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_1975/jobhistory/job/job_1457613852865_1975

我用过这个：

#!/bin/bash


STARTTIME="Tue Apr 12 10:13:01 EDT 2016"
ENDTIME="Tue Apr 12 10:13:59 EDT 2016"

start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')

echo "start=$start :: end=$end"

mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

有一份额外的工作：

job_1457613852865_9785  SUCCEEDED       1460470436726       yyyyyyyyyy    nnnnnnnnnn       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://888888.xxxxxxxxxx.com:8088/proxy/application_1457613852865_9785/jobhistory/job/job_1457613852865_9785

Answer 1

您可以将日期转换为millisec值，然后使用awk过滤数据：

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"

start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')"

echo "start=$start :: end=$end"
mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

Bash - 基于时间戳的搜索，以毫秒为单位

1 个答案: