我有以下代码,我试图根据给定的grok模式解析日志行。我使用concurrent.futures.ProcessPoolExecutor进行并行调用。程序执行正常,但输出未按预期打印。
这是我的代码:
from concurrent.futures import ProcessPoolExecutor, as_completed
from korg import LineGrokker, PatternRepo
pr = PatternRepo()
patterns = [
'%{COMBINEDAPACHELOG}',
'%{COMMONAPACHELOG}',
'%{HTTPD_ERRORLOG}'
]
text = [
'37.162.60.195 - - [06/Jun/2018:17:31:29 -0400] "PUT /app/main/posts HTTP/1.0" 200 5055 "http://harris-johnson.com/main/register/" "Mozilla/5.0 (Windows NT 5.0) AppleWebKit/5311 (KHTML, like Gecko) Chrome/13.0.850.0 Safari/5311"',
'::1 - - [26/Dec/2016:16:16:29 +0200] "GET /favicon.ico HTTP/1.1" 404 209',
'[Mon Dec 26 16:22:08 2016] [error] [client 192.168.33.1] File does not exist: /var/www/favicon.ico'
]
pool = ProcessPoolExecutor(len(patterns))
def grok_parser(pattern, text):
return LineGrokker(pattern, pr).grok(text)
def parse_type(ia):
futures = [pool.submit(grok_parser, pattern, ia) for pattern in patterns]
for r in as_completed(futures):
return r.result()
for i in text:
print(parse_type(i))
输出继电器:
{'clientip': '37.162.60.195', 'ident': '-', 'auth': '-', 'timestamp': '06/Jun/2018:17:31:29 -0400', 'verb': 'PUT', 'request': '/app/main/posts', 'httpversion': '1.0', 'rawrequest': None, 'response': '200', 'bytes': '5055'}
None
None
预期产出:
{'clientip': '37.162.60.195', 'ident': '-', 'auth': '-', 'timestamp': '06/Jun/2018:17:31:29 -0400', 'verb': 'PUT', 'request': '/app/main/posts', 'httpversion': '1.0', 'rawrequest': None, 'response': '200', 'bytes': '5055', 'referrer': '"http://harris-johnson.com/main/register/"', 'agent': '"Mozilla/5.0 (Windows NT 5.0) AppleWebKit/5311 (KHTML, like Gecko) Chrome/13.0.850.0 Safari/5311"'}
{'clientip': '::1', 'ident': '-', 'auth': '-', 'timestamp': '26/Dec/2016:16:16:29 +0200', 'verb': 'GET', 'request': '/favicon.ico', 'httpversion': '1.1', 'rawrequest': None, 'response': '404', 'bytes': '209'}
{'timestamp': 'Mon Dec 26 16:22:08 2016', 'loglevel': 'error', 'clientip': '192.168.33.1', 'message': 'File does not exist: /var/www/favicon.ico', 'module': None, 'pid': None, 'tid': None, 'proxy_errorcode': None, 'proxy_message': None, 'clientport': None, 'errorcode': None}
如何从上面的代码获得所需?任何帮助表示赞赏。