如何在日志文件中格式化日期和时间输出,并从输出中删除重复条目

时间:2016-05-17 08:39:26

标签: bash shell unix awk grep

我从几个不同的日志文件中输出。我使用下面的命令从日志文件中过滤掉。我需要按日期和时间输出过滤器,所有输出应按日期和时间按排序顺序显示。并且所有结果应合并到一个文件中。我正在搜索具有@ABC和@DEF的分析用户,并删除重复的条目。

cat log_file1.log | grep 'Profiling user :' | awk '{print $1,$21}' | cut -c2- | grep '@ABC'

log_file1.log的输出

    2016-05-16T10:50:09.170+00:00] ABHIJEET.VADGAMA@ABC.com
    2016-05-16T10:50:09.170+00:00] ABHIJEET.VADGAMA@ABC.com
    2016-05-16T10:56:43.944+00:00] AAMIR.SHAIKH@ABC.com
    2016-05-16T10:56:43.944+00:00] AAMIR.SHAIKH@ABC.com
    2016-05-16T11:00:55.961+00:00] SALIL.WAGLE@ABC.com
    2016-05-16T11:00:55.961+00:00] SALIL.WAGLE@ABC.com
    2016-05-16T11:12:23.326+00:00] ABHIJEET.VADGAMA@ABC.com
    2016-05-16T11:12:23.326+00:00] ABHIJEET.VADGAMA@ABC.com
    2016-05-16T11:26:35.311+00:00] VIPIN.SHARMA@ABC.com
    2016-05-16T11:26:35.311+00:00] VIPIN.SHARMA@ABC.com
    2016-05-16T12:08:33.719+00:00] PRITHA.MAZUMDAR@ABC.com
    2016-05-16T12:08:33.719+00:00] PRITHA.MAZUMDAR@ABC.com
    2016-05-16T12:58:40.453+00:00] VIPIN.SHARMA@ABC.com
    2016-05-16T12:58:40.453+00:00] VIPIN.SHARMA@ABC.com
    2016-05-16T13:44:45.657+00:00] PRITHA.MAZUMDAR@ABC.com
    2016-05-16T13:44:45.657+00:00] PRITHA.MAZUMDAR@ABC.com
    2016-05-16T14:42:35.818+00:00] HOLLY.M.JOHNSONSTUHR@ABC.com
    2016-05-16T14:42:35.818+00:00] HOLLY.M.JOHNSONSTUHR@ABC.com
    2016-05-16T14:46:07.353+00:00] CLEAVAN.R.DSOUZA@JPMORGAN.COM

log_file2.log

的输出
2016-05-12T08:16:49.438+00:00] SABEEN.RAZA@ABC.com
2016-05-12T08:16:49.438+00:00] SABEEN.RAZA@ABC.com
2016-05-12T08:32:52.239+00:00] AKPOME.O.OMORHIRHI@ABC.com
2016-05-12T08:32:52.239+00:00] AKPOME.O.OMORHIRHI@ABC.com
2016-05-12T08:52:19.074+00:00] PUNIT.X.KHIRODWALA@JPMORGAN.COM
2016-05-12T08:52:19.074+00:00] PUNIT.X.KHIRODWALA@JPMORGAN.COM
2016-05-12T09:39:48.475+00:00] GRAZYNA.M.JANKOWSKA@JPMORGAN.COM
2016-05-12T09:39:48.475+00:00] GRAZYNA.M.JANKOWSKA@JPMORGAN.COM
2016-05-12T09:58:42.659+00:00] PRIYANKA.V.GAIKWAD@ABC.com
2016-05-12T09:58:42.659+00:00] PRIYANKA.V.GAIKWAD@ABC.com
2016-05-12T11:45:58.756+00:00] ABHIJEET.VADGAMA@ABC.com
2016-05-12T11:45:58.756+00:00] ABHIJEET.VADGAMA@ABC.com
2016-05-12T12:36:02.116+00:00] SHANE.L.BERGENSTEN@ABC.com
2016-05-12T12:36:02.116+00:00] SHANE.L.BERGENSTEN@ABC.com
2016-05-12T13:03:52.589+00:00] VIPIN.SHARMA@ABC.com
2016-05-12T13:03:52.589+00:00] VIPIN.SHARMA@ABC.com

cat logfile_3.log | grep'性能分析用户:' | awk' {print $ 1,$ 21}' |切-c2- | grep' @ DEF'

2016-05-12T13:50:40.132+00:00] PRIYANKA.V.GAIKWAD@DEF.com
2016-05-12T13:50:40.132+00:00] PRIYANKA.V.GAIKWAD@DEF.com
2016-05-12T14:06:32.173+00:00] VIPIN.SHARMA@DEF.com
2016-05-12T14:06:32.173+00:00] VIPIN.SHARMA@DEF.com

我需要输出如下: -

  Date       Time               User 
    2016-05-16  8:16:49     SABEEN.RAZA@ABC.com
    2016-05-16  8:16:49     SABEEN.RAZA@ABC.com
    2016-05-16  8:32:52     AKPOME.O.OMORHIRHI@ABC.com
    2016-05-16  8:32:52     AKPOME.O.OMORHIRHI@ABC.com
    2016-05-16  8:52:19     PUNIT.X.KHIRODWALA@JPMORGAN.COM
    2016-05-16  8:52:19     PUNIT.X.KHIRODWALA@JPMORGAN.COM
    2016-05-16  9:39:48     GRAZYNA.M.JANKOWSKA@JPMORGAN.COM
    2016-05-16  9:39:48     GRAZYNA.M.JANKOWSKA@JPMORGAN.COM
    2016-05-16  9:58:42     PRIYANKA.V.GAIKWAD@ABC.com
    2016-05-16  9:58:42     PRIYANKA.V.GAIKWAD@ABC.com
    2016-05-16  11:45:58    ABHIJEET.VADGAMA@ABC.com
    2016-05-16  11:45:58    ABHIJEET.VADGAMA@ABC.com
    2016-05-16  12:36:02    SHANE.L.BERGENSTEN@ABC.com
    2016-05-16  12:36:02    SHANE.L.BERGENSTEN@ABC.com
    2016-05-16  13:03:52    VIPIN.SHARMA@ABC.com
    2016-05-16  13:03:52    VIPIN.SHARMA@ABC.com
    2016-05-16  13:50:40    PRIYANKA.V.GAIKWAD@DEF.com
    2016-05-16  13:50:40    PRIYANKA.V.GAIKWAD@DEF.com
    2016-05-16  14:06:32    VIPIN.SHARMA@DEF.com
    2016-05-16  14:06:32    VIPIN.SHARMA@DEF.com
    2016-05-16  10:50:09    ABHIJEET.VADGAMA@ABC.com
    2016-05-16  10:56:43    AAMIR.SHAIKH@ABC.com
    2016-05-16  10:56:43    AAMIR.SHAIKH@ABC.com
    2016-05-16  10:50:09    ABHIJEET.VADGAMA@ABC.com
    2016-05-16  11:00:55    SALIL.WAGLE@ABC.com
    2016-05-16  11:00:55    SALIL.WAGLE@ABC.com
    2016-05-16  11:12:23    ABHIJEET.VADGAMA@ABC.com

1 个答案:

答案 0 :(得分:1)

这是部分解决方案,使用合理的日期格式

/Profiling user :/ && ($21 ~ "@ABC.com" || $21 ~ "@DEF.com") {
    split($1, time, "[^0-9]");
    print time[4] "/" time[3] "/" time[2] "\t" $21;
}