我有一个如下所示的日志:
2014-06-12T14:59:01.120997 - MONITOR - Load [0.01] Spawn [9]
2014-06-12T15:00:01.187993 - MONITOR - Load [0.67] Spawn [7]
2014-06-12T15:01:01.292163 - MONITOR - Load [0.86] Spawn [0]
2014-06-12T15:02:01.409863 - PROVISION - [other line format]
2014-06-12T15:02:03.305833 - MONITOR - Load [0.09] Spawn [8]
我有logstash监控它并使用此输入和过滤器发送到elasticsearch(更有效的方式来写这个也很感激。我对logstash / elasticsearch来说是全新的):
input {
file {
type => "load_monitor"
path => "/var/log/load_monitor.log"
sincedb_path => ["/opt/load_monitor"]
}
}
filter {
if [type] == "load_monitor" {
grok {
match => ["message", "%{TIMESTAMP_ISO8601:logtime} - %{WORD:montype} - %{GREEDYDATA:content}"]
}
if [montype] == "MONITOR" {
grok {
match => ["content", "Load \[%{NUMBER:load:float}\] Spawn \[%{NUMBER:spawn:int}\]"]
}
mutate {
remove_field => ['content']
}
}
if [montype] == "PROVISION" {
// do other stuff
}
}
}
通过kibana在弹性搜索中似乎是正确的(加载和生成字段存在且填充正确):
{
"message":"2014-06-12T15:15:01.436632 - MONITOR - Load [0.71] Spawn [5]",
"@version":"1",
"@timestamp":"2014-06-12T22:15:02.304Z",
"type":"load_monitor",
"host":"prx01",
"path":"/var/log/load_monitor.log",
"logtime":"2014-06-12T15:15:01.436632",
"montype":"MONITOR",
"load":0.71,
"spawn":5
}
当我尝试在" spawn"的kibana中创建直方图时价值,"产生"不是自动建议的,当我尝试绘制spawn时,我得到了这个错误:
SearchParseException[
[logstash-2014.06.12][4]: from[-1],size[-1]:
Parse Failure [
Failed to parse source
[
{
"facets": {
"0": {
"date_histogram": {"key_field":"@timestamp","value_field":"spawn","interval":"10s"},
"global":true,
"facet_filter": {
"fquery":{
"query":{
"filtered":{
"query":{
"query_string": {
"query":"type: \"load_monitor\""
}
},
"filter":{
"bool":{
"must":[
{
"range":{
"@timestamp":{"from":1402610796579,"to":1402611696579}
}
},
{
"fquery":{
"query":{
"query_string":{
"query":"type:(\"load_monitor\")"
}
},
"_cache":true
}
}
]
}
}
}
}
}
}
}
},
"size":0
}
]
]
]
任何人都可以帮忙吗?我怀疑我必须告诉elasticsearch以某种方式搜索spawn字段,但我不确定如何。