Question

我正在使用Filebeat - ＆gt; Logstash - ＆gt; Elasticsearch - ＆gt; Kibana概述了我的glassfish日志文件。

这是我的Filebeat配置。 http://pastie.org/10958269

############################# Filebeat ######################################
filebeat:
  # List of prospectors to fetch data.
  prospectors:
    paths:
    - /Users/mmlug/Documents/univeristy/runtime/glassfish-3.1/glassfish/domains/domain1/logs/server.log
input_type: log
exclude_lines: ["^DBG"]

multiline:
  pattern: (?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:Log Level}\|%{DATA:server_version}\|%{JAVACLASS:Class}\|%{DATA:thread}\|%{DATA:message_detail}\|\#\]
  negate: true
  match: after

  output:
  ### Logstash as output
  logstash:
     # The Logstash hosts
  hosts: ["localhost:5044"]

  logging:

  shipper:

我的日志消息看起来像这样。

[#|2016-11-01T11:29:33.347+0100|INFO|glassfish3.1.2|com.MachineProxy|_ThreadID=226;_ThreadName=Thread-2;|proxy started|#]

我目前的模式看起来像这样。我已经使用http://grokdebug.herokuapp.com/

进行了测试

(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL}\|%{DATA:server_version}\|%{JAVACLASS}\|%{DATA:thread}\|%{DATA:message_detail}\|\#\]

我希望在Kibana中有一个字段用于“DateTime”，“Log Level”，“Server version”，“Java Class Name”，“Thread”和“Message detail”

2016/11/01 14:59:10.672190 publish.go:109: DBG  Publish: {
 "@timestamp": "2016-11-01T14:59:04.606Z",
 "beat": {
 "hostname": "localhost",
 "name": "localhost"
 },
 "count": 1,
"fields": null,
"input_type": "log",
"message": "[#|2016-11-    01T15:59:02.991+0100|INFO|glassfish3.1.2|com.MachineProxy|_ThreadID=227;_ThreadName=Thread-2;|MachineProxy touched.|#]\n",
"offset": 1671903,
"source": "../logs/server.log",
"type": "log"
}

我在ELK泊坞窗容器中的Logstash配置。

root@4569cf1f66ab:/etc/logstash/conf.d# 
01-lumberjack-input.conf.backup  02-beats-input.conf  10-syslog.conf  11-   nginx.backup  30-output.conf

02-节拍-input.conf中

input {
  beats {
    port => 5044
    ssl => false
    #ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt"
    #ssl_key => "/etc/pki/tls/private/logstash-beats.key"
  }
}

10的syslog.conf

filter {
 if [type] == "log" {
  grok {
   match => { "message", "(?m)\[\#\|%{TIMESTAMP_ISO8601:timestamp}\|%{LOGLEVEL:Log Level}\|%{DATA:server_version}\|%{JAVACLASS:Class}\|%{DATA:thread}\|%{DATA:message_detail}\|\#\]" }
  add_field => [ "Log level", "%{LOGLEVEL:Log Level}" ]
}
syslog_pri { }
date {
  match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
}

非常感谢提前。

托马斯

如何添加logstash grok过滤器和字段

0 个答案: