Question

我正在使用 filebeat 6.2.1 来获取日志，解析为JSON并将其发送到 elasticsearch 6.5.4 ，但出现以下错误：

WARN    elasticsearch/client.go:520
Cannot index event publisher.Event{Content:beat.Event{Timestamp: 
[...]
"mapper_parsing_exception","reason":
"object mapping for [event] tried to parse field [event] as object, but found a concrete value

我认为这是一个字段映射问题，并且与嵌套的json解析有关，但是我不确定如何解决它。

filebeat.yml：

# ======================================
# Filebeat prospectors

filebeat.prospectors:

- type: log

# Change to true to enable this prospector configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
 - /var/log/bridge/*.json
 #- c:\programdata\elasticsearch\logs\*

 ignore_older: 24h
 scan_frequency: ${FilebeatScanFrequency}

 json:
   message_key: event
   keys_under_root: true

# ======================================
# Elasticsearch template setting 

setup.template.settings:
index.number_of_shards: 1

filebeat输出示例：

{
  "_index": "filebeat-6.2.1-date",
  "_type": "doc",
  "_id": "nh6Z9WcBwAmR4kjdfofdk-H",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2018-12-28T16:15:48.012Z",
    "event": "",
    "Timestamp": "2018-12-28T21:45:42.7876369+05:30",
    "Level": "Information",
    "MessageTemplate": "{AuthenticationScheme} was not authenticated. 
Failure message: {FailureMessage}",
    "beat": {
      "name": "DB-01",
      "hostname": "DB-01",
      "version": "6.2.1"
    },
    "Properties": {
      "EventId": {
        "Id": 7
      },
      "Country": "Canada",
      "Format": "json",
      "MachineName": "DB-01",
      "ThreadId": 57,
      "Source": "Example API",
      "FailureMessage": "No authorization header.",
      "SourceContext": 
      "Odachi.AspNetCore.Authentication.Basic.BasicMiddleware",
      "RequestPath": "/status",
      "AuthenticationScheme": "Basic",
      "ProcessId": 4627,
      "EnvironmentUserName": "EXAMPLEDNS\\example.api",
      "RequestId": "0NKDFUJDFKDL",
      "Environment": "Production"
    },
    "source": "D:\\Logs\\Company Example-api.json",
    "offset": 9575044,
    "RenderedMessage": "\"Basic\" was not authenticated. Failure message: 
\"No authorization header.\"",
    "prospector": {
      "type": "log"
    }
  },
  "fields": {
    "@timestamp": [
      "2018-12-28T16:15:48.012Z"
    ]
  },
  "sort": [
    154601321324346
  ]
}

我正在寻找一种正确解决映射错误并成功将已解析的JSON日志发送到elasticsearch的方法。

更新：我想用一系列可能的solutions（弹性团队成员）Steffen Siering提到的问题来更新此问题：

您实际上正在使用filebeat 6.5.x
您可以将filebeat 6.2.x数据索引到6.5.x使用的相同索引中
您可以将filebeat 6.2.x数据索引到与模板匹配的索引中
以前/其他6.5.x安装中的配置
您的json文件中的类型并不总是字符串

就我而言，我已将Filebeat升级到最新的6.5.x，这导致了事件字段的映射错误。为了解决这个问题，我将filebeat降级到6.2.x，从而解决了该问题。

如何修复elasticsearch filebeat映射器解析异常-尝试将field [x]解析为对象，但找到了具体值？

0 个答案: