我想通过logstash解析netflow我使用cisco开关将netflow发送到logstash,然后将流数据推送到elasticsearch,但它似乎无法工作,这是我的logstash配置:
input{
udp{
host => "120.126.160.91"
port => 5556
codec => netflow
type =>"netflow"
}
}
filter{
if[type]=="netflow"
{
if "" not in [IPV4_SRC_ADDR] and "" not in [IPV6_SRC_ADDR]
{
drop{}
}
}
}
output{
elasticsearch {
hosts => ["120.126.160.91:9200"]
}
if[type]=="netflow"
{
stdout{codec=> rubydebug}
}
}
但似乎有些不对劲。 logstash日志:
Sending Logstash's logs to D:/ELK/logstash-6.1.1/logs which is now configured via log4j2.properties
[2018-01-31T17:24:52,309][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"D:/ELK/logstash-6.1.1/modules/fb_apache/configuration"}
[2018-01-31T17:24:52,324][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"D:/ELK/logstash-6.1.1/modules/netflow/configuration"}
[2018-01-31T17:24:52,651][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-01-31T17:24:53,763][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.1.1"}
[2018-01-31T17:24:54,481][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-01-31T17:25:08,371][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://120.127.160.91:9200/]}}
[2018-01-31T17:25:29,250][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://120.127.160.91:9200/, :path=>"/"}
[2018-01-31T17:25:29,515][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://120.127.160.91:9200/"}
[2018-01-31T17:25:29,578][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>nil}
[2018-01-31T17:25:29,598][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-01-31T17:25:29,621][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-01-31T17:25:29,638][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-01-31T17:25:29,701][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//120.127.160.91:9200"]}
[2018-01-31T17:25:29,748][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>16, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>2000, :thread=>"#<Thread:0x3d774198 run>"}
[2018-01-31T17:25:29,842][INFO ][logstash.pipeline ] Pipeline started {"pipeline.id"=>"main"}
[2018-01-31T17:25:29,920][INFO ][logstash.inputs.udp ] Starting UDP listener {:address=>"120.127.160.91:5556"}
[2018-01-31T17:25:29,951][INFO ][logstash.inputs.udp ] UDP listener started {:address=>"120.127.160.91:5556", :receive_buffer_bytes=>"65536", :queue_size=>"2000"}
[2018-01-31T17:25:30,013][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}