如何用awk或sed拆分日志文件。替换python脚本

时间:2018-03-12 08:12:40

标签: python awk sed

让我们假设您每天都有一个仪器日志文件。白天可能会发生几次重启。出于某种原因,您希望每次重启都有一个文件。

最后我使用python来做,但我想用awk或sed做同样的事情。 请让我知道你的想法。

python script split_instrument_log.py

def split_instrument_log(filename):
    first_line = '--- ServiceHost Start ---'
    count = 0
    with open(filename, 'r') as handle:
        text = handle.read()
        split_text = text.split('\n' + first_line)
        for split in split_text:
            split_file_name = filename + "." + str(count)
            with open(split_file_name, 'w') as split_handle:
                if count > 0:
                    split_handle.write(first_line)
                split_handle.write(split)
            count = count + 1

filename = "instrument.log";
split_instrument_log(filename)

示例instrument.log:

--- ServiceHost Start ---
11:43:54.745 00000001 HOST I  Creating System 2/19/2018 11:43:54 AM
...
--- ServiceHost Start ---
14:47:37.071 00000001 HOST I  Creating System 2/19/2018 2:47:37 PM
...
--- ServiceHost Start ---
18:27:57.463 00000001 HOST I  Creating System 2/19/2018 6:27:57 PM
...

result instrument.log.0

--- ServiceHost Start ---
11:43:54.745 00000001 HOST       I  Creating System 2/19/2018 11:43:54 AM
...

我有另一个日志,它以时间戳和地址开头,例如

[05/02/2018 13:32:30.160 UTC] Main Thread (0xb4692000)/ 0 INF socMainExecutable

如何更新awk脚本,但请注意时间戳和地址不是常量?

1 个答案:

答案 0 :(得分:1)

使用awk非常直接:

<强> INPUT:

$ more instrument.log
--- ServiceHost Start ---
11:43:54.745 00000001 HOST I  Creating System 2/19/2018 11:43:54 AM
blabla1
blabla2
blabla3
...
--- ServiceHost Start ---
14:47:37.071 00000001 HOST I  Creating System 2/19/2018 2:47:37 PM
...
blabla4
blabla5
blabla6
--- ServiceHost Start ---
18:27:57.463 00000001 HOST I  Creating System 2/19/2018 6:27:57 PM
...
blabla7
blabla8
blabla9

awk脚本:

awk -v i=-1 '/--- ServiceHost Start ---/{i++; print $0 > "instrument.log."i; next}{print $0 >> "instrument.log."i}' instrument.log

<强>输出:

$ more instrument.log.?
::::::::::::::
instrument.log.0
::::::::::::::
--- ServiceHost Start ---
11:43:54.745 00000001 HOST I  Creating System 2/19/2018 11:43:54 AM
blabla1
blabla2
blabla3
...
::::::::::::::
instrument.log.1
::::::::::::::
--- ServiceHost Start ---
14:47:37.071 00000001 HOST I  Creating System 2/19/2018 2:47:37 PM
...
blabla4
blabla5
blabla6
::::::::::::::
instrument.log.2
::::::::::::::
--- ServiceHost Start ---
18:27:57.463 00000001 HOST I  Creating System 2/19/2018 6:27:57 PM
...
blabla7
blabla8
blabla9

<强>解释

  • -v i=-1将变量i传递给awk,初始值为-1,您也可以在BEGIN子句中定义它,如下所示:BEGIN{i=-1}
  • /--- ServiceHost Start ---/{i++; print $0 > "instrument.log."i; next}每当awk找到包含--- ServiceHost Start ---的行时,它会增加i并将行内容打印到文件"instrument.log."i,然后再转到下一行。 (如果文件存在,它将覆盖该文件)
  • {print $0 >> "instrument.log."i}其他行只是附加到文件"instrument.log."i