我们一直在运行ElasticSearch v.1.6.0和Kibana v.4.1.0,NGINX作为代理生产大约两个月,并且设置处理多个应用程序的记录。我们有830.000个文档,总共450MB和一个节点。所有日志语句都使用Serilog发送到ElasticSearch,后者每天创建一个索引。
当我运行像
这样的查询时 curl.exe -XGET '[hostname]:9200/serilog-2016.02.20/_search
?q=*&sort=@timestamp:desc&size=500'
然后我立刻得到了结果。
但是,如果我在过去的15分钟内查询Kibana,那么NGINX会在请求等待2分钟(每次TTFB超过2分钟)后返回Bad Gateway,表示发生了超时。
Kibana请求
POST http://[hostname]/elasticsearch/_msearch?timeout=0&ignore_unavailable=true&preference=1455968700774
请求有效负载
{
"index": "serilog-2016.02.20",
"ignore_unavailable": true
}{
"size": 500,
"sort": [{
"@timestamp": {
"order": "desc",
"unmapped_type": "date"
}
}],
"highlight": {
"pre_tags": ["@kibana-highlighted-field@"],
"post_tags": ["@/kibana-highlighted-field@"],
"fields": {
"*": {
}
},
"fragment_size": 2147483647
},
"aggs": {
"2": {
"date_histogram": {
"field": "@timestamp",
"interval": "30s",
"pre_zone": "+01:00",
"pre_zone_adjust_large_interval": true,
"min_doc_count": 0,
"extended_bounds": {
"min": 1455967802350,
"max": 1455968702350
}
}
}
},
"query": {
"filtered": {
"query": {
"match_all": {
}
},
"filter": {
"bool": {
"must": [{
"range": {
"@timestamp": {
"gte": 1455967802352,
"lte": 1455968702352
}
}
}],
"must_not": []
}
}
}
},
"fields": ["*",
"_source"],
"script_fields": {
},
"fielddata_fields": ["fields.effectiveDate",
"fields.assignIpPortRequestModel.NeededFrom",
"fields.ipPorts.LastCancelled.State.DateTime",
"fields.returnedOrganizations.DateCreated",
"fields.provisionings.CreatedAt",
"fields.ProductParameterData.StartDate",
"fields.provisioningEvent.UpdatedAt",
"fields.networkConfig.Expiration",
"fields.returnedOrganizations.DateModified",
"fields.provisioningEvent.CreatedAt",
"fields.ipPorts.State.ActiveFrom.DateTime",
"fields.ptpPorts.LastCancelled.State.DateTime",
"fields.portReservationData.ExpirationDate",
"fields.productInstanceDtoList.StartDate",
"@timestamp",
"fields.ipPorts.State.LastCancelled.DateTime",
"fields.productInstanceWithProductDtos.StartDate",
"fields.neededFrom",
"fields.portNeededFrom.State.DateTime",
"fields.products.StartDate",
"fields.productList.StartDate",
"fields.networkReservation.Expiration",
"fields.portReservationData.Products.StartDate"]
}
上周我试图找出导致糟糕表现的原因。我将Kibana中的索引模式从serilog- *修改为[serilog-] YYYY.MM.DD(在4.1.0版中不是默认值),但这没有帮助。