Question

我在filebeat.yml中使用Filebeat多行模式，它从单个文件中获取输入，如下所示：

＆＃13;

2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

＆＃13;

filebeat.yml

＆＃13;

multiline:
pattern: Identifier
negate: true
match: after

＆＃13;

我使用上面的配置来匹配标识符＆＃39;在线。输出应符合要求

＆＃13;

event -1 :
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

event -2 :
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
	ContentType: text/xml; charset=utf-8
	ContextPath: 
	LocalAddr: 
	LocalName: 
	PathInfo: 
	PathTranslated: 
	QueryString: 
	RequestURI: 
	RequestURL: 
	RemoteHost: 
	ServletPath: 
	Header: Host: 
	Header: Content-Length: 
	Header: Accept-Encoding: 
	Header: SOAPAction: ""
	Header: User-Agent: Apache-HttpClient/4.2.1 
	Header: Content-Type: text/xml; charset=utf-8
	Header: Connection: Keep-Alive
	Header: Accept: text/xml
	
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
<env:Envelope></env:Envelope>

＆＃13;

Answer 1

根据您的示例输入，我们可以使用包含requestStartIdentifier: Identifier的行来表示新事件的开始。我使用https://play.golang.org/p/BZ2ujeOZZ-来测试不同的多行参数。

Filebeat config：

filebeat:
  prospectors:
    - input_type: log
      paths:
        - input.txt
      multiline:
        pattern: 'requestStartIdentifier: Identifier$'
        negate:  true
        match:   after

output:
  console:
    pretty: true

Filebeat输出（扩展换行符）：

{
  "@timestamp": "2016-10-06T21:51:27.244Z",
  "beat": {
    "hostname": "host",
    "name": "host"
  },
  "input_type": "log",
  "message": "2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
    ContentType: text/xml; charset=utf-8
    ContextPath: 
    LocalAddr: 
    LocalName: 
    PathInfo: 
    PathTranslated: 
    QueryString: 
    RequestURI: 
    RequestURL: 
    RemoteHost: 
    ServletPath: 
    Header: Host: 
    Header: Content-Length: 
    Header: Accept-Encoding: 
    Header: SOAPAction: \"\"
    Header: User-Agent: Apache-HttpClient/4.2.1 
    Header: Content-Type: text/xml; charset=utf-8
    Header: Connection: Keep-Alive
    Header: Accept: text/xml

2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
\u003cenv:Envelope\u003e\u003c/env:Envelope\u003e
",
  "offset": 962,
  "source": "input.txt",
  "type": "log"
}
{
  "@timestamp": "2016-10-06T21:51:27.244Z",
  "beat": {
    "hostname": "host",
    "name": "host"
  },
  "input_type": "log",
  "message": "2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestStartIdentifier: Identifier
2016-10-06 14:36:00.419 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : requestUri: 
2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : HttpServletRequest:
    ContentType: text/xml; charset=utf-8
    ContextPath: 
    LocalAddr: 
    LocalName: 
    PathInfo: 
    PathTranslated: 
    QueryString: 
    RequestURI: 
    RequestURL: 
    RemoteHost: 
    ServletPath: 
    Header: Host: 
    Header: Content-Length: 
    Header: Accept-Encoding: 
    Header: SOAPAction: \"\"
    Header: User-Agent: Apache-HttpClient/4.2.1 
    Header: Content-Type: text/xml; charset=utf-8
    Header: Connection: Keep-Alive
    Header: Accept: text/xml

2016-10-06 14:36:00.420 DEBUG 29695 --- [XNIO-2 task-2] c.a.a.s.endpoint.endone.server  : uri: , request:
\u003cenv:Envelope\u003e\u003c/env:Envelope\u003e",
  "offset": 1923,
  "source": "input.txt",
  "type": "log"
}

模式匹配Filebeat多行模式中的完整单词

1 个答案: