我正在尝试使用Kibana
显示我的数据文件我的档案格式如下
timeStamp;elapsed;label;responseCode;responseMessage;threadName;success;failureMessage;bytes;grpThreads;allThreads;Latency;SampleCount;ErrorCount;Hostname
2016-01-16 02:27:17,565;912;HTTP Request;200;OK;Thread Group 1-5;true;;78854;10;10;384;1;0;sundeep-Latitude-E6440 timeStamp;elapsed;label;responseCode;responseMessage;threadName;success;failureMessage;bytes;grpThreads;allThreads;Latency;SampleCount;ErrorCount;Hostname
2016-01-16 02:27:17,565;912;HTTP Request;200;OK;Thread Group 1-5;true;;78854;10;10;384;1;0;sundeep-Latitude-E6440
要映射上述数据,我的logstash配置如下:
input {
file {
path => [ "/home/sundeep/data/test.csv"]
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
if ([message] =~ "responseCode") {
drop { }
} else {
csv {
separator => ";"
columns => ["timeStamp", "elapsed", "label", "responseCode","responseMessage","threadName",
"success","failureMessage", "bytes", "grpThreads", "allThreads", "Latency",
"SampleCount", "ErrorCount", "Hostname"]
}
}
}
output {
elasticsearch { hosts => ["localhost:9200"]
index => "aa-%{+yyyy-MM-dd}"
}
}
模板文件如下:
{
"template": "aa-*",
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0,
"index.refresh_interval": "5s"
},
"mappings": {
"logs": {
"properties": {
"timeStamp": {
"index": "analyzed",
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss,SSS"
},
"elapsed": {
"type": "long"
},
"dummyfield": {
"type": "long"
},
"label": {
"type": "string"
},
"responseCode": {
"type": "integer"
},
"threadName": {
"type": "string"
},
"success": {
"type": "boolean"
},
"failureMessage":{
"type": "string"
},
"bytes": {
"type": "long"
},
"grpThreads": {
"type": "long"
},
"allThreads": {
"type": "long"
},
"Latency": {
"type": "long"
},
"SampleCount": {
"type": "long"
},
"ErrorCount": {
"type": "long"
},
"Hostname": {
"type": "string"
}
}
}
}
}
现在您可以看到,只要我使用配置文件启动logstash,就会在elasticsearch中创建一个新索引。
新创建的索引从预期的aa- *开始。
现在,我在Kibana中搜索索引,我可以看到如下:
[
但是,当我尝试绘制折线图时,我看不到任何数据。
此外,我能够通过此博客文章可视化另一个数据集:enter link description here
跟踪日志:
[2016-01-16 02:45:41,105] [INFO] [cluster.metadata] [Hulk 2099] [aa-2016-01-15]删除索引 [2016-01-16 02:46:01,370] [INFO] [cluster.metadata] [Hulk 2099] [aa-2016-01-15]创建索引,导致[auto(bulk api)],模板[aa],分片1 / [0],映射[日志] [2016-01-16 02:46:01,451] [INFO] [cluster.metadata] [Hulk 2099] [aa-2016-01-15] update_mapping [logs]
ELK Stack ElasticSearch - 2.1 Logstash - 2.1 Kibana - 4.3.1.1