我希望有人能够帮助我解决这个问题。 我有点卡在regex / grok / logstash语法上。 我试图解析kibana日志，但日志中的字段并不总是以相同的顺序。
这里有3个日志作为示例。
{"name":"Kibana","hostname":"22e1923b59f3","pid":1,"level":30,"req":{"method":"POST","url":"/elasticsearch/_msearch?timeout=0&ignore_unavailable=true&preference=1438549037831","headers":{"host":"192.168.33.10:5601","connection":"keep-alive","content-length":"779","accept":"application/json, text/plain, */*","origin":"http://192.168.33.10:5601","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36","content-type":"application/json;charset=UTF-8","referer":"http://192.168.33.10:5601/","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.8,fr;q=0.6","cookie":"PHPSESSID=nhbu44n9jce33312je8346gm43; fastcoinlang=en; csrftoken=RAI1SVRsAGPmE490nhObxX9M1WYFuFjn"},"remoteAddress":"192.168.33.1","remotePort":56797},"res":{"statusCode":200,"responseTime":180,"contentLength":158203},"msg":"POST /_msearch?timeout=0&ignore_unavailable=true&preference=1438549037831 200 - 180ms","time":"2015-08-02T21:02:50.359Z","v":0}
{"name":"Kibana","hostname":"22e1923b59f3","pid":1,"level":30,"req":{"method":"GET","url":"/worker-json.js","headers":{"host":"192.168.33.10:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36","accept":"*/*","referer":"http://192.168.33.10:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8,fr;q=0.6","cookie":"PHPSESSID=nhbu44n9jce33312je8346gm43; fastcoinlang=en; csrftoken=RAI1SVRsAGPmE490nhObxX9M1WYFuFjn","if-none-match":"W/\"10ca1-14e40807fc0\"","if-modified-since":"Mon, 29 Jun 2015 18:07:20 GMT"},"remoteAddress":"192.168.33.1","remotePort":56797},"res":{"statusCode":304,"responseTime":1,"contentLength":0},"msg":"GET /worker-json.js 304 - 1ms","time":"2015-08-02T21:02:59.102Z","v":0}
{"name":"Kibana","hostname":"a274fbd7485b","pid":1,"level":30,"req":{"method":"GET","url":"/bower_components/font-awesome/fonts/fontawesome-webfont.woff?v=4.2.0","headers":{"host":"192.168.33.10:5601","connection":"keep-alive","cache-control":"max-age=0","if-none-match":"W/\"ffac-14e40807bd8\"","if-modified-since":"Mon, 29 Jun 2015 18:07:19 GMT","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.125 Safari/537.36","accept":"*/*","referer":"http://192.168.33.10:5601/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8,fr;q=0.6","cookie":"PHPSESSID=nhbu44n9jce33312je8346gm43; fastcoinlang=en; csrftoken=RAI1SVRsAGPmE490nhObxX9M1WYFuFjn"},"remoteAddress":"192.168.33.1","remotePort":59999},"res":{"statusCode":304,"responseTime":2,"contentLength":0},"msg":"GET /bower_components/font-awesome/fonts/fontawesome-webfont.woff?v=4.2.0 304 - 2ms","time":"2015-08-02T22:55:19.190Z","v":0}
到目前为止我可以解析的内容：
{\"name\":%{QS:name},\"hostname\":%{QS:hostname},\"pid\":%{INT:pid},\"level\":%{INT:level},\"req\":{\"method\":%{QS:req_method},\"url\":%{QS:req_url},\"headers\":{\"host\":%{QS:headers_host},\"connection\":%{QS:headers_connection}(,\"content-length\":%{QS:headers_content_length}|)(,\"accept\":%{QS:headers_accept}|)(,\"cache-control\":%{QS:headers_cache_control}|)(,\"if-none-match\":%{QS:headers_if_none_match}|)(,\"if-modified-since\":%{QS:headers_if_modified_since}|)(,\"origin\":%{QS:headers_origin}|)(,\"user-agent\":%{QS:headers_user_agent}|)(,\"accept\":%{QS:headers_accept}|)(,\"content-type\":%{QS:headers_content_type}|)(,\"referer\":%{QS:headers_referer}|)(,\"accept-encoding\":%{QS:headers_accept_encoding}|)(,\"accept-language\":%{QS:headers_accept_language}|)(,\"cookie\":%{QS:headers_cookie}|)},\"remoteAddress\":%{QS:remoteaddress},\"remotePort\":%{INT:remoteport}},\"res\":{\"statusCode\":%{INT:res_statuscode},\"responseTime\":%{INT:res_responsetime},\"contentLength\":%{INT:res_contentlength}},\"msg\":%{QS:res_msg},\"time\":\"%{TIMESTAMP_ISO8601:time}\",\"v\":%{INT:v}}
有了这个，我可以注意一个字段是否缺失，但是当一个字段出现在另一个字段之前/之后时。如何捕捉而不管理现场秩序？
如果您对如何提高语法效率有任何提示，我也会采取这种方法！