Question

我搜索了针对此问题的解决方案，并添加了Ruby代码。问题是如果我使用gsub我也会收到错误。

我需要一个解决方案来解决Ruby错误或其他方法来为转义字符添加异常。

这是我的Logstash过滤器：

filter {
  if [type] == "apache" {
    json {
          source => "message"
        }
    ruby {
          code => "str=event['url'];  str=str.gsub('\\','\*'); event['url']=str;"
        }
 }
}

我以json格式从Apache获取我的Logs。以下是Logstash错误消息：

 {:timestamp=>"2016-06-28T12:49:21.821000+0100", :message=>"Error
 parsing json", :source=>"message", :raw=>"{ \"@version\": \"1\",
 \"@timestamp\": \"2016-06-28T12:49:16.000+0100\", \"message\": \"GET
 /yolo/q-ap%C3%Ablah/?search%5Bdist%5D=15&page=13 HTTP/1.1\", \"via\":
 \"192.168.220.100\", \"client-ip\": \"123.123.123.123\",
 \"remote-logname\": \"-\", \"remote-user\": \"-\", \"recv-time\":
 \"[28/Jun/2016:12:49:16 +0100]\", \"serve-time-microsec\": \"88613\",
 \"request\": \"GET /yolo/q-ap%C3%Ablah/?search%5Bdist%5D=15&page=13
 HTTP/1.1\", \"status\": \"200\", \"size\": \"184985\", \"referer\":
 \"-\", \"user-agent\": \"Mozilla/5.0 (compatible; Googlebot/2.1;
 +http://www.google.com/bot.html)\", \"url\": \"/yolo/q-ap\\xc3\\xablah/\", \"query\":
 \"?search%5Bdist%5D=15&page=13\", \"method\": \"GET\", \"protocol\":
 \"HTTP/1.1\", \"vhost\": \"www.site.com\", \"received-size\": \"1136\"
 }", :exception=>#LogStash::Json::ParserError: Unrecognized character
 escape 'x' (code 120) at Source: [B@2ebf051e; line: 1, column: 599>,
 :level=>:warn}

 {:timestamp="2016-06-28T12:49:21.821000+0100", :message=>"Ruby
 exception occurred: undefined method `gsub' for nil:NilClass",
 :level=>:error}

Answer 1

感谢logstash团队，我能够解决我的问题。 https://discuss.elastic.co/t/add-execption-for-special-character-from-json-log/54285/3

这是正确的配置：

filter {
  if [type] == "apache" {

    ruby {
        code => " if event['message']
                  event['message'] = event['message'].gsub('\x','Xx')
                  event['message'] = event['message'].gsub('\\x','XXx')
                  end
                "
            }

    json {
          source => "message"
        }

 }
}

请注意，ruby需要高于json格式。

Answer 2

Logstash 5对event['message']的使用有重大变化。 https://github.com/logstash-plugins/logstash-filter-aggregate/issues/48#issuecomment-258297534

对我有用的配置是：

ruby {
  code => " if event.get('message')
               event.set('message', event.get('message').gsub('\x','Xx'))
               event.set('message', event.get('message').gsub('\\x','XXx'))
            end
          "
}

Logstash - 为json转义字符添加execption

2 个答案: