Question

我在为Cloud Foundry设置Logstash方面遇到了一些问题我曾经看到的任何来源都指示我进行以下配置

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}
filter {
if [@type] in ["syslog", "relp"] {
  # Parse Cloud Foundry logs from loggregator (syslog)
  # see https://github.com/cloudfoundry/loggregator/blob/master/src/loggregator/sinks/syslogwriter/syslog_writer.go#L156

  grok {
      match => { "syslog_procid" => "\[(?<log_source>[^/\]]+)(?:/(?<log_source_id>[^\]]+))?\]" }
      tag_on_failure => [
          "fail/logsearch-for-cloudfoundry/loggregator/_grokparsefailure"
      ]
  }

  if !("fail/logsearch-for-cloudfoundry/loggregator/_grokparsefailure" in [tags]) {
      #If it looks like JSON, it must be JSON...
      if [syslog_message] =~ /^\s*{".*}\s*$/ {
          json {
              source => "syslog_message"
          }

          # @todo seems like some messages have @timestamp in them? seems ci-specific
          date {
              match => [ "@timestamp", "ISO8601" ]
          }
      } else {
          mutate {
              add_field => [ "message", "%{syslog_message}" ]
          } 
          if [message] == "-" {
              mutate {
                  remove_field => "message"
              } 
          }
      }

      mutate {
          rename => [ "syslog_program", "@source.app_id" ]
      }

      mutate {
          add_tag => "cloudfoundry_loggregator"
          remove_field => "syslog_facility"
          remove_field => "syslog_facility_code"
          remove_field => "syslog_message"
          remove_field => "syslog_severity"
          remove_field => "syslog_severity_code"
          remove_field => "syslog5424_ver"
          remove_field => "syslog6587_msglen"
      }
  }

} 
}
output {

  stdout { codec => rubydebug }
}

但是没有一个过滤器与Cloud Foundry的日志匹配，我可以获得如下的日志

2015-08-03T09:51:15.000+00:00 [RTR] OUT mm1-spring-music.example.com - [03/08/2015:09:51:15 +0000] "GET /assets/templates/grid.html HTTP/1.1" 200 1450 "http://mm1-spring-music.example.com/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36" 10.10.125.30:37611 x_forwarded_for:"Xx.XX, 0.0.0.0" vcap_request_id:ae307d85-01c3-433b-487d-92d897dbcf99 response_time:0.002201911 app_id:08be9fc8-c7a3-4613-bf12-1a9c7d98cc27

KIndly建议是否已存在一些可用于使用Logstash进行解析的模式

Answer 1

确保您真正在云代工厂部署清单中使用relp协议。过滤器if [@type] in ["syslog", "relp"]正在测试该类型。

如果您使用普通的UDP syslog或TCP，请从过滤器中删除该子句。您可以在kibana中执行此操作，方法是转到settings-＆gt; objects-＆gt;搜索并禁用该过滤器或删除relp条件。

CloudFoundry Loggregator的Logstash配置

1 个答案: