我确实通过多行编解码器从rsyslog(systemd)解析了一条消息,但是它无法正常工作。
我使用这种模式
pattern => "^%{SYSLOGBASE} %{TIMESTAMP_ISO8601}"
解析消息的开头:
Sep 28 14:42:10 ibcore9t tarantool[148700]: 2018-09-28 14:42:10.805
完整的消息是这样的:
Oct 01 10:27:21 ibcore9t tarantool[148700]: 2018-10-01 10:27:21.533 [148700] main/14364/http/172.25.8.18:64209 init.lua:179 E> Pipeline execution failed: Function "focus_spotforward_couponpayment_map" in pipeline "focus_spotforward_couponpayment_handle": ./ib-common/sandbox.lua:130: Trade not found Oct 01 10:27:21 ibcore9t tarantool[148700]: stack traceback: Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:37: in function 'call_function' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:75: in function 'call_pipeline' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:111: in function Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'pcall' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/pipeline/init.lua:109: in function 'call' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-core/server.lua:119: in function Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'xpcall' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/errors.lua:93: in function 'pcall' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-core/server.lua:162: in function Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'xpcall' Oct 01 10:27:21 ibcore9t tarantool[148700]: ... Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/errors.lua:93: in function 'pcall' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/funcall.lua:41: in function 'call' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./t-connect/soapserver.lua:139: in function 'callback' Oct 01 10:27:21 ibcore9t tarantool[148700]: ./ib-common/http.lua:201: in function 'sub' Oct 01 10:27:21 ibcore9t tarantool[148700]: ...e/tarantool/core0/.rocks/share/tarantool/http/server.lua:610: in function Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'pcall' Oct 01 10:27:21 ibcore9t tarantool[148700]: ...e/tarantool/core0/.rocks/share/tarantool/http/server.lua:687: in function 'process_client' Oct 01 10:27:21 ibcore9t tarantool[148700]: ...e/tarantool/core0/.rocks/share/tarantool/http/server.lua:1132: in function Oct 01 10:27:21 ibcore9t tarantool[148700]: [C]: in function 'pcall' Oct 01 10:27:21 ibcore9t tarantool[148700]: builtin/socket.lua:1059: in function
但是我只看到基巴纳语的第一行
Pipeline execution failed: Function "focus_spotforward_couponpayment_map" in pipeline "focus_spotforward_couponpayment_handle": ./ib-common/sandbox.lua:130: Trade not found
我的logstash.conf:
input {
udp {
port => 5000
type => syslog
codec => multiline {
pattern => "^%{SYSLOGBASE} %{TIMESTAMP_ISO8601}"
negate => true
what => "next"
}
}
}
filter {
grok {
match => [ "message", "%{SYSLOGBASE} %{TIMESTAMP_ISO8601:timestamp_tarantool} \[%{NUMBER:id}\] %{WORD}/%{NUMBER}/%{WORD:method} %{WORD:log_level}> %{GREEDYDATA:msg}",
"message", "%{SYSLOGBASE} %{TIMESTAMP_ISO8601:timestamp_tarantool} \[%{NUMBER:id}\] %{WORD}/%{NUMBER}/%{WORD:method}/%{IP}:%{NUMBER} %{WORD:log_level}> %{GREEDYDATA:msg}",
"message", "%{SYSLOGBASE} %{TIMESTAMP_ISO8601:timestamp_tarantool} \[%{NUMBER:id}\] %{WORD}/%{NUMBER}/%{WORD:method}/%{IP}:%{NUMBER} %{WORD}.%{WORD}:%{NUMBER} %{WORD:log_level}> %{GREEDYDATA:msg}"
]
}
mutate {
replace => { "message" => "%{msg}" }
remove_field => [ "msg" ]
}
if "_grokparsefailure" in [tags] { drop {} }
if [message] =~ "queue" {
mutate {
add_field => { "repair_queue" => "%{message}"}
}
}
}
output {
elasticsearch {
action => "index"
index => "logstash-%{+YYYY.MM.dd}"
hosts => "ibmetrics2:9200"
}
}
请帮助