Filebeat输入中的多行模式(Python错误回溯)中的空行未正确解析?

时间:2019-12-30 13:11:42

标签: logstash filebeat logstash-file

应该收集并作为单行发布到logstash的日志行:

[pid: 17318|app: 0|req: 1/2] 10.14.206.28 (jaavedkhan) {60 vars in 1296 bytes} [Mon Dec 30 15:51:38 2019] GET /en/ => generated 27 bytes in 711 msecs (HTTP/1.1 500) 6 headers in 316 bytes (1 switches on core 0)
Mon Dec 30 15:51:39 2019 - announcing my loyalty to the Emperor...


Internal Server Error: /en/
Traceback (most recent call last):
  File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 124, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "./core/views.py", line 31, in get
    1/0
ZeroDivisionError: division by zero

我的文件拍配置:

filebeat:
  inputs:
    - type: log
      paths:
        - "/var/log/uwsgi/vassals/dsr-incentives.log"
      fields_under_root: true
      multiline:
        pattern: '\[pid:\s*\d*\|app:'
        negate: true
        match: after
      fields:
        log_type: app-access
        appserver: uwsgi
        app: dsr-incentives
        server_name: server-name.domain.com

我用日志行检查了多行模式https://play.golang.org

enter image description here 结果与预期的一样,但是收割机在“ Internat服务器错误”处拆分了日志行

    Publish event: {
  "@timestamp": "2019-12-30T13:02:56.564Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.5.1"
  },
  "log": {
    "offset": 128736,
    "file": {
      "path": "/var/log/uwsgi/vassals/dsr-incentives.log"
    },
    "flags": [
      "multiline"
    ]
  },
  "appserver": "uwsgi",
  "server_name": "xyz",
  "log_type": "app-access",
  "host": {
    "name": "xyz"
  },
  "agent": {
    "hostname": "apps-1",
    "id": "d3417bc3-213c-4d5e-a9b5-2273178262d0",
    "version": "7.5.1",
    "name": "xyz",
    "type": "filebeat",
    "ephemeral_id": "125578d6-44d1-4103-94bc-a1d062091487"
  },
  "message": "Internal Server Error: /en/\nTraceback (most recent call last):\n  File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/exception.py\", line 34, in inner\n    response = get_response(request)\n  File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 126, in _get_response\n    response = self.process_exception_by_middleware(e, request)\n  File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 124, in _get_response\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\n  File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py\", line 68, in view\n    return self.dispatch(request, *args, **kwargs)\n  File \"/opt/dsr-incentives/venv/lib/python3.5/site-packages/django/views/generic/base.py\", line 88, in dispatch\n    return handler(request, *args, **kwargs)\n  File \"./core/views.py\", line 31, in get\n    1/0\nZeroDivisionError: division by zero",
  "tags": [
    "filebeat"
  ],
  "input": {
    "type": "log"
  },
  "app": "dsr-incentives",
  "ecs": {
    "version": "1.1.0"
  }
}

我认为问题是多行被“内部服务器错误”之前的日志中出现的空行分隔。

更新 当“内部服务器错误”上方没有空行时,日志消息将被正确解析

0 个答案:

没有答案