应该收集并作为单行发布到logstash的日志行:
[pid: 17318|app: 0|req: 1/2] 10.14.206.28 (jaavedkhan) {60 vars in 1296 bytes} [Mon Dec 30 15:51:38 2019] GET /en/ => generated 27 bytes in 711 msecs (HTTP/1.1 500) 6 headers in 316 bytes (1 switches on core 0)
Mon Dec 30 15:51:39 2019 - announcing my loyalty to the Emperor...
Internal Server Error: /en/
Traceback (most recent call last):
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 68, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 88, in dispatch
return handler(request, *args, **kwargs)
File "./core/views.py", line 31, in get
1/0
ZeroDivisionError: division by zero
我的文件拍配置:
filebeat:
inputs:
- type: log
paths:
- "/var/log/uwsgi/vassals/dsr-incentives.log"
fields_under_root: true
multiline:
pattern: '\[pid:\s*\d*\|app:'
negate: true
match: after
fields:
log_type: app-access
appserver: uwsgi
app: dsr-incentives
server_name: server-name.domain.com
我用日志行检查了多行模式https://play.golang.org:
结果与预期的一样,但是收割机在“ Internat服务器错误”处拆分了日志行
Publish event: {
"@timestamp": "2019-12-30T13:02:56.564Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.5.1"
},
"log": {
"offset": 128736,
"file": {
"path": "/var/log/uwsgi/vassals/dsr-incentives.log"
},
"flags": [
"multiline"
]
},
"appserver": "uwsgi",
"server_name": "xyz",
"log_type": "app-access",
"host": {
"name": "xyz"
},
"agent": {
"hostname": "apps-1",
"id": "d3417bc3-213c-4d5e-a9b5-2273178262d0",
"version": "7.5.1",
"name": "xyz",
"type": "filebeat",
"ephemeral_id": "125578d6-44d1-4103-94bc-a1d062091487"
},
"message": "Internal Server Error: /en/\nTraceback (most recent call last):\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/exception.py\", line 34, in inner\n response = get_response(request)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 126, in _get_response\n response = self.process_exception_by_middleware(e, request)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 124, in _get_response\n response = wrapped_callback(request, *callback_args, **callback_kwargs)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py\", line 68, in view\n return self.dispatch(request, *args, **kwargs)\n File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py\", line 88, in dispatch\n return handler(request, *args, **kwargs)\n File \"./core/views.py\", line 31, in get\n 1/0\nZeroDivisionError: division by zero",
"tags": [
"filebeat"
],
"input": {
"type": "log"
},
"app": "dsr-incentives",
"ecs": {
"version": "1.1.0"
}
}
我认为问题是多行被“内部服务器错误”之前的日志中出现的空行分隔。
更新 当“内部服务器错误”上方没有空行时,日志消息将被正确解析