我正在尝试使用geoip和流利的nginx日志。 至于nginx,我正在使用以下过滤器:
<filter *.nginx>
@type record_transformer
enable_ruby
<record>
log ${ if ( !record['log'].match(/"upstream_cache_status": "HIT"/).nil? ) then record['log'].gsub(',"upstream_response_time ":','') else record['log'] end }
@timestamp ${require 'time'; a=record['log'].match( /\"time_local\": \"([0-9.]*)\"/); if !a.nil? then Time.at(a[1].to_f).utc.iso8601(3) else record['@timestamp'] end }
</record>
</filter>
<filter *.nginx>
@type parser
format json
key_name log
reserve_data true
</filter>
对于流利的geoip,我用过:
<filter *.nginx>
@type geoip
geoip_lookup_key true_client_ip
enable_key_country_code geoip_country
enable_key_city geoip_city
enable_key_latitude geoip_lat
enable_key_longitude geoip_lon
flush_interval 5s
</filter>
因此我得到以下输出:
{
"_index": ".hicham",
"_type": "forever",
"_id": "AWJOoZ1cPXjXRJAPgU2P",
"_version": 1,
"_score": null,
"_source": {
"log": "IP - - [22/Mar/2018:16:53:54 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"curl/7.47.0\" \"-\"\n",
"stream": "stdout",
"container_name": "nginx",
"@service": "nginxService",
"@timestamp": "2018-03-22T16:53:54.939Z",
},
"fields": {
"@timestamp": [
1521737634939
]
}
}
因此输出中没有geoip字段。
有没有解决方案?
答案 0 :(得分:1)
您的配置看起来应该是这样的,以便geoip过滤器正常工作
<filter nginx.access>
@type geoip
geoip_lookup_keys true_client_ip
# geoip2_database "/path/to/your/GeoLite2-City.mmdb" (using bundled GeoLite2-City.mmdb by default)
# Specify backend library (geoip2_c, geoip, geoip2_compat)
backend_library geoip2_c
<record>
city ${city.names.en["true_client_ip"]}
latitude ${location.latitude["true_client_ip"]}
longitude ${location.longitude["true_client_ip"]}
country ${country.iso_code["true_client_ip"]}
country_name ${country.names.en["true_client_ip"]}
region_name ${subdivisions.0.names.en["true_client_ip"]}
# GeoJSON (lat lon as array) is useful for Kibana's bettermap.
# ex. [-122.05740356445312, 37.4192008972168]
location_array '[${location.longitude["true_client_ip"]},${location.latitude["true_client_ip"]}]'
</record>
# To avoid get stacktrace error with `[null, null]` array for elasticsearch.
skip_adding_null_record true
flush_interval 5s
</filter>