2016-06-02 17:00:32<ERROR><CommonsHTTPTransportSender:361>:
org.apache.abcd2.abcdFault
at org.apache.abcd2.abcdFault.makeFault(abcdFault.java:430)
at org.apache.abcd2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:83)
at org.apache.abcd2.transport.http.CommonsHTTPTransportSender.sendUsingOutputStream(CommonsHTTPTransportSender.java:358)
at java.lang.Thread.run(Thread.java:636)
Caused by: com.my.application.IOException: null
at com.ctc.wstx.sw.BaseStreamWriter.finishDocument(BaseStreamWriter.java:1692)
at com.ctc.wstx.sw.BaseStreamWriter.close(BaseStreamWriter.java:288)
at org.apache.axiom.util.stax.wrapper.XMLStreamWriterWrapper.close(XMLStreamWriterWrapper.java:46)
at org.apache.abcd2.WriteTo(SOAPMessageFormatter.java:79)
... 27 more
所以现在如果我想得到像LOGLEVEL类名这样的值,并且由于我得到了loglevel和类名的值而没有得到引起的消息那么......怎么可能?
以下是我的配置文件。
input{
file{
path => "D:\Log\application.log"
start_position => beginning
codec => multiline{
pattern => "%{TIMESTAMP_ISO8601}"
what => "next"
negate => true
}
}
}
filter{
grok{
match => ["message","^%{TIMESTAMP_ISO8601}<%{LOGLEVEL}><(?<JavaClass>.*[:].*)>"]
}
mutate {
gsub => ['message', "\n", ""]
gsub => ['message', "\t", ""]
}
}
output {
stdout { }
elasticsearch {
index => "ABCD_%{+YYYY.MM.dd}"
}
}
我主要关注的是解析timestamp loglevel classname和由值引起的
答案 0 :(得分:0)
您的多行编解码器错误,这就是您的配置无效的原因。 (我测试过了)。
当我在我的盒子上使用它时,这是你的配置的stdout(你没有发布):
artur@pandaadb:~/dev/logstash$ ./logstash-2.3.2/bin/logstash -f conf2/
Settings: Default pipeline workers: 8
Pipeline main started
{
"@timestamp" => "2016-06-22T09:19:01.896Z",
"message" => "2016-06-02 17:00:32<ERROR><CommonsHTTPTransportSender:361>: ",
"@version" => "1",
"path" => "/home/artur/tmp/logstash/in2/test.log",
"host" => "pandaadb",
"JavaClass" => "CommonsHTTPTransportSender:361"
}
查看您的消息甚至没有任何信息?没有什么可以匹配的,因为你的mutliline不起作用。我希望这首先是一个问题:
所以这是一个工作配置(我使用的是多行过滤器而不是编解码器):
multiline {
pattern => "^%{TIMESTAMP_ISO8601}"
negate => true
what => "previous"
}
有了这个,我可以更新你的grok以接受原因:
grok {
match => ["message","^%{TIMESTAMP_ISO8601:ts}<%{LOGLEVEL:log}><(?<JavaClass>.*[:].*)>.*Caused by:%{GREEDYDATA:data}"]
}
在我的盒子上输入你的输入后,我得到:
artur@pandaadb:~/dev/logstash$ ./logstash-2.3.2/bin/logstash -f conf2/
Settings: Default pipeline workers: 8
Defaulting pipeline worker threads to 1 because there are some filters that might not work with multiple worker threads {:count_was=>8, :filters=>["multiline"], :level=>:warn}
Pipeline main started
{
"message" => "2016-06-02 17:00:32<ERROR><CommonsHTTPTransportSender:361>: \norg.apache.abcd2.abcdFault\n at org.apache.abcd2.abcdFault.makeFault(abcdFault.java:430)\n at org.apache.abcd2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:83)\n at org.apache.abcd2.transport.http.CommonsHTTPTransportSender.sendUsingOutputStream(CommonsHTTPTransportSender.java:358)\n at java.lang.Thread.run(Thread.java:636)\nCaused by: com.my.application.IOException: null\n at com.ctc.wstx.sw.BaseStreamWriter.finishDocument(BaseStreamWriter.java:1692)\n at com.ctc.wstx.sw.BaseStreamWriter.close(BaseStreamWriter.java:288)\n at org.apache.axiom.util.stax.wrapper.XMLStreamWriterWrapper.close(XMLStreamWriterWrapper.java:46)\n at org.apache.abcd2.WriteTo(SOAPMessageFormatter.java:79)\n ... 27 more",
"@version" => "1",
"@timestamp" => "2016-06-22T09:22:38.227Z",
"path" => "/home/artur/tmp/logstash/in2/test.log",
"host" => "pandaadb",
"tags" => [
[0] "multiline"
],
"ts" => "2016-06-02 17:00:32",
"log" => "ERROR",
"JavaClass" => "CommonsHTTPTransportSender:361",
"data" => " com.my.application.IOException: null\n at com.ctc.wstx.sw.BaseStreamWriter.finishDocument(BaseStreamWriter.java:1692)\n at com.ctc.wstx.sw.BaseStreamWriter.close(BaseStreamWriter.java:288)\n at org.apache.axiom.util.stax.wrapper.XMLStreamWriterWrapper.close(XMLStreamWriterWrapper.java:46)\n at org.apache.abcd2.WriteTo(SOAPMessageFormatter.java:79)\n ... 27 more"
}
希望有所帮助。
为了将来参考,关于logstash问题,有一个示例将stdin和stdout打印到stdout总是有帮助的,因为它可以很快地再现。
stdout(rubydebug)的输出也会告诉你原始消息的确切内容,并且很容易看出多行不起作用,这也是导致问题的原因。
干杯!
阿图尔