通过logstash将netflow解析为elasticsearch

时间:2018-01-31 09:51:25

标签: elasticsearch logstash

我想通过logstash解析netflow我使用cisco开关将netflow发送到logstash,然后将流数据推送到elasticsearch,但它似乎无法工作,这是我的logstash配置:

input{
    udp{
        host => "120.126.160.91"
        port => 5556
        codec => netflow
        type =>"netflow"

    }
}
    filter{
        if[type]=="netflow"
        {
            if "" not in [IPV4_SRC_ADDR] and "" not in [IPV6_SRC_ADDR]
            {
                drop{}
            }
        }
    }
output{
elasticsearch {
                hosts => ["120.126.160.91:9200"]
            }
    if[type]=="netflow"
    {
        stdout{codec=> rubydebug}
    }
}

但似乎有些不对劲。 logstash日志:

Sending Logstash's logs to D:/ELK/logstash-6.1.1/logs which is now configured via log4j2.properties
[2018-01-31T17:24:52,309][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"D:/ELK/logstash-6.1.1/modules/fb_apache/configuration"}
[2018-01-31T17:24:52,324][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"D:/ELK/logstash-6.1.1/modules/netflow/configuration"}
[2018-01-31T17:24:52,651][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-01-31T17:24:53,763][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.1.1"}
[2018-01-31T17:24:54,481][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-01-31T17:25:08,371][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://120.127.160.91:9200/]}}
[2018-01-31T17:25:29,250][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://120.127.160.91:9200/, :path=>"/"}
[2018-01-31T17:25:29,515][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://120.127.160.91:9200/"}
[2018-01-31T17:25:29,578][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>nil}
[2018-01-31T17:25:29,598][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-01-31T17:25:29,621][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-01-31T17:25:29,638][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-01-31T17:25:29,701][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//120.127.160.91:9200"]}
[2018-01-31T17:25:29,748][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>16, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>2000, :thread=>"#<Thread:0x3d774198 run>"}
[2018-01-31T17:25:29,842][INFO ][logstash.pipeline        ] Pipeline started {"pipeline.id"=>"main"}
[2018-01-31T17:25:29,920][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"120.127.160.91:5556"}
[2018-01-31T17:25:29,951][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"120.127.160.91:5556", :receive_buffer_bytes=>"65536", :queue_size=>"2000"}
[2018-01-31T17:25:30,013][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}

0 个答案:

没有答案