Question

我尝试使用GROK定义模式方法将非结构化数据拆分为结构化数据，我在logstash配置文件中定义了如下所示的模式。

else if [fields][index] == "secure1"
{
   grok
   {
      match => [
               "message", "%{SYSLOGTIMESTAMP:syslog_date} %{SYSLOGHOST:syslog_host} %{DATA:syslog_program}(\[%{POSINT:pid}\])\: %{WORD:login} password for %{USERNAME:username} from %{IP:ip} %{GREEDYDATA}",
               "message", "%{SYSLOGTIMESTAMP:syslog_date_session_start} %{SYSLOGHOST:syslog_host} %{DATA:syslog_program}(\[%{POSINT:pid}\])\: %{GREEDYDATA}\)\: session %{WORD:mode} for user %{USERNAME:username} %{GREEDYDATA}"
               ]
   }

   date
   {
      match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
   }
}

为了匹配以下日志文件：

========
Jan 25 12:44:57 wqprod01 sshd[28509]: Accepted password for 548201 from XXXXXXXXX port 55446 ssh2
Jan 25 12:44:57 wqprod01 sshd[28509]: pam_unix(sshd:session): session opened for user 548201 by (uid=0)
Jan 25 12:53:10 wqprod01 su: pam_unix(su:session): session closed for user root
Jan 25 12:53:31 wqprod01 sudo:   619565 : TTY=pts/0 ; PWD=/home/619565 ; USER=root ; COMMAND=/bin/chown 619565:619565 fortigate-fw.log
Jan 25 12:53:49 wqprod01 sshd[28668]: Accepted password for 619565 from XXXXXXXXXX port 55724 ssh2
Jan 25 12:53:49 wqprod01 sshd[28668]: pam_unix(sshd:session): session opened for user 619565 by (uid=0)
Jan 25 12:53:49 wqprod01 sshd[28678]: subsystem request for sftp
Jan 25 12:54:21 wqprod01 sshd[28690]: Accepted password for 619565 from 10.212.134.200 port 55734 ssh2
Jan 25 12:54:21 wqprod01 sshd[28690]: pam_unix(sshd:session): session opened for user 619565 by (uid=0)
Jan 25 12:54:21 wqprod01 sshd[28700]: subsystem request for sftp
Jan 25 12:55:30 wqprod01 sshd[28690]: pam_unix(sshd:session): session closed for user 619565
Jan 25 12:56:57 wqprod01 sshd[28509]: pam_unix(sshd:session): session closed for user 548201

我能够看到日志已正确编入索引。但是我在logstash文件中定义的任何字段在Kibana中都不可见。下面是json文件。

{
  "_index": "secure1-2018.03.02",
  "_type": "doc",
  "_id": "EkNA5WEB-2UZLW5hSDDW",
  "_version": 1,
  "_score": null,
  "_source": {
    "beat": {
      "name": "CTSC00973598401",
      "hostname": "CTSC00973598401",
      "version": "6.1.2"
    },
    "offset": 103,
    "@version": "1",
    "tags": [
      "beats_input_codec_plain_applied"
    ],
    "fields": {
      "index": "secure1"
    },
    "message": "Jan 25 12:44:57 wqprod01 sshd[28509]: Accepted password for 548201 from XXXXXXXXX port 55446 ssh2",
    "@timestamp": "2018-03-02T05:47:28.706Z",
    "source": "/var/log/secure1",
    "host": "CTSC00973598401"
     },
  "fields": {
    "@timestamp": [
      "2018-03-02T05:47:28.706Z"
      ]
     },
  "sort": [
    1519969648706
  ]
}

你能帮我解决这个问题。我在这里错过了什么吗？

GROK定义的模式在kibana中不可见

0 个答案: