Question

我正在尝试将系统日志消息转发到Logstash，然后转发到Elasticsearch。

在源系统上，我创建了/etc/rsyslog.d/logstash.conf：

*.* action(type="omfwd" target="elk.example.com" port="50513" protocol="tcp")

在elk.example.com上，Logstash配置非常基本：

input {
  tcp {
    port => 50513
    type => syslog
  }
}

output {
  elasticsearch { hosts => ["localhost:9200"] }
}

在源系统上运行logger hello之后，我在Kibana中看到的消息：

{
  "_index": "logstash-2019.03.12",
  "_type": "doc",
  "_id": "0DWkcWkB9fKcBjDya4El",
  "_version": 1,
  "_score": null,
  "_source": {
    "host": "domotique.example.com",
    "message": [
      "<13>Mar 12 12:23:14 domotique root: hello",
      "hello"
    ],
    "@timestamp": "2019-03-12T11:23:14.475Z",
    "@version": "1",
    "port": 43752,
    "timestamp": "Mar 12 12:23:14",
    "logsource": "domotique",
    "program": "root",
    "type": "syslog"
  },
  "fields": {
    "@timestamp": [
      "2019-03-12T11:23:14.475Z"
    ]
  },
  "sort": [
    1552389794475
  ]
}

为什么hello被重复（并添加了逗号）？

这本身就是一个问题。一个必然的结论是系统日志格式未被识别，但是我补充说thanks to another question

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }
}

作为过滤器，可以解决此问题（请参见下面的"syslog_message": "hello"）

{
  "_index": "logstash-2019.03.12",
  "_type": "doc",
  "_id": "fDWtcWkB9fKcBjDynoPf",
  "_version": 1,
  "_score": null,
  "_source": {
    "timestamp": "Mar 12 12:33:17",
    "syslog_pri": "13",
    "type": "syslog",
    "@version": "1",
    "host": "domotique.example.com",
    "logsource": "domotique",
    "program": "root",
    "@timestamp": "2019-03-12T11:33:17.534Z",
    "syslog_program": "root",
    "port": 52636,
    "syslog_message": "hello",
    "syslog_hostname": "domotique",
    "message": [
      "<13>Mar 12 12:33:17 domotique root: hello",
      "hello"
    ],
    "syslog_timestamp": "Mar 12 12:33:17"
  },
  "fields": {
    "@timestamp": [
      "2019-03-12T11:33:17.534Z"
    ]
  },
  "highlight": {
    "syslog_message.keyword": [
      "@kibana-highlighted-field@hello@/kibana-highlighted-field@"
    ],
    "message": [
      "<13>Mar 12 12:33:17 domotique root: @kibana-highlighted-field@hello@/kibana-highlighted-field@",
      "@kibana-highlighted-field@hello@/kibana-highlighted-field@"
    ],
    "syslog_message": [
      "@kibana-highlighted-field@hello@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1552390397534
  ]
}

仍然：为什么邮件一开始是重复的？

为什么syslog的“消息”字段重复？

0 个答案: