标签使用GROK删除了CWS / Scansafe的标签

时间:2018-11-12 19:10:42

标签: elastic-stack logstash-grok grok

我是GROK的新手,我有一个ELK堆栈想将一些CWS / Scansafe日志摄取到该婴儿中。我一直无法获得良好的解析输出。我正在使用http://grokdebug.herokuapp.com/grokconstructor.appspot.com/do/match,试图获得正确的解析经历了一次地狱。日志为W3C,但由制表符\t分隔。我建立了这种模式,但我无法正确解析多字字段(具有空格)以及空字段(并非每条记录都有每条输出)。任何帮助打破这一点或指出正确方向的帮助将不胜感激。

Here is the pattern:

%{TIMESTAMP_ISO8601:cws_datatime} %{WORD:timezone}\t%{IPV4:c_ip}\t%{NUMBER:x_ss_company_id}\t(%{IPV4:cs_X_Forwarded_For})?(\t%{NOTSPACE:cs_username})?(\t%{WORD:cs_method})?(\t%{WORD:cs_uri_scheme})?(\t%{NOTSPACE:cs_host})?(\t%{NUMBER:cs_uri_port})?(\t%{NOTSPACE:cs_uri_path})?(\t%{DATA:cs_uri_query})?(\t%{DATA:user_agent})?(\t%{DATA:cs_Content_Type})?(\t%{NUMBER:cs_bytes})?(\t%{NUMBER:sc_bytes})?(\t%{NUMBER:sc_status})?(\t%{DATA:sc_Content_Type})?(\t%{IPV4:s_ip})?(\t%{NOTSPACE:x_ss_category})?(\t%{WORD:x_ss_last_rule_name})?(\t%{WORD:x_ss_last_rule_action})?(\t%{WORD:x_ss_block_type})?(\t%{WORD:x_ss_block_value})?\t%{IPV4:x_ss_external_ip}(\t%{NOTSPACE:x_ss_referer_host})?

以下是一些示例日志:

#Fields: datatime   c-ip    x-ss-company-id cs(X-Forwarded-For) cs-username cs-method   cs-uri-scheme   cs-host cs-uri-port cs-uri-path cs-uri-query    cs(User-Agent)  cs(Content-Type)    cs-bytes    sc-bytes    sc-status   sc(Content-Type)    s-ip    x-ss-category   x-ss-last-rule-name x-ss-last-rule-action   x-ss-block-type x-ss-block-value    x-ss-external-ip    x-ss-referer-host
2018-11-07 15:40:28 GMT 10.11.77.96 2100000000      10.10.77.96 CONNECT https   d33t3vvu2t2yu5.cloudfront.net   443 /       Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; MS-RTC LM 8; InfoPath.3; )  -   952 4982    0       13.32.243.161   c:infr  default allow           9.99.239.26 
2018-11-07 15:40:28 GMT 10.11.167.50    2100000000      WinNT://local\bobby CONNECT https   pbs.twimg.com   443 /       Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0;  rv:11.0) like Gecko   -   2034    873898  0       12.21.91.70 c:snet  default allow           9.99.239.26 
2018-11-07 15:40:30 GMT 10.79.239.26    2100000000      9.99.239.26 CONNECT https   assets.customer.com 443 /       Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36   -   1006    8508    0       9.99.187.71 c:busi  default allow           9.99.239.26 
2018-11-07 15:40:35 GMT 10.79.239.26    2100000000      9.99.239.26 CONNECT https   cloud-ec-asn.amp.cisco.com  443 /           -   2754    7316    0       52.70.59.121    c:comp  default allow           9.99.239.26 
2018-11-07 15:40:36 GMT 10.79.239.26    2100000000      9.99.239.26 CONNECT https   inetupload.indsci.com   443 /           -   1589    7038    0       198.187.140.58  c:busi  default allow           9.99.239.26 
2018-11-07 15:40:44 GMT 10.79.239.26    2100000000      9.99.239.26 CONNECT https   api.bing.com    443 /       Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko    -   0   0   0       13.107.5.80 c:srch  default allow           9.99.239.26 
2018-11-07 15:41:03 GMT 10.14.144.19    2100000000      WinNT://local\jane  CONNECT https   www.bing.com    443 /       Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0;  rv:11.0) like Gecko   -   22552   35245   0       13.107.21.200   c:srch  default allow           9.99.239.26 
2018-11-07 15:41:04 GMT 10.79.239.26    2100000000      9.99.239.26 CONNECT https   cloud-ec-asn.amp.cisco.com  443 /           -   13742   9326    0       52.70.47.45 c:comp  default allow           9.99.239.26 
2018-11-07 15:41:04 GMT 10.79.239.26    2100000000      9.99.239.26 GET http    ocsp.verisign.com   80  /STAJBgUrDgMCGgUABBTSqZMG5M8TA9rdzkbCnNwuMAd5VgQUz5mp6nsm9EvJjo%2FX8AUm7%2BPSp50CEF7OSr0FTbDuGs71fE0%2FRCw%3D       Microsoft-CryptoAPI/6.1 -   0   1660    200 application/ocsp-response   23.4.187.27 c:csec  default allow           9.99.239.26 
2018-11-07 15:40:57 GMT 10.79.239.26    2100000000      9.99.239.26 GET http    ourtripguide.pro    80  /35/wp-content/uploads/2018/10/10-Things-You-Need-To-Do-After-Checking-Into-A-Hotel-Room.jpg        Outlook-iOS/696.1208477.prod.iphone (2.103.0)   -   0   0   0       185.133.39.50   unclassified    webrep  block   webrep  Reputation-Viruses  9.99.239.26 

0 个答案:

没有答案