Logastash CSV过滤器|转换

时间:2016-04-27 18:54:56

标签: csv logstash elastic-stack

面临从csv转换某些值的问题。 如果我只声明一个转换,例如' =>'整数'它工作正常。 如果我在下面的配置中添加另一个转换,它会崩溃....

    input {
        tcp {
            port => 5000
        }
        file {
              path => '/tmp/logstash/databases/1.csv'
              type => 'test'
              start_position => 'beginning'
          }
    }

    filter {
        csv {
            columns => ['Id','ElapsedRealTimeNanos','Accuracy','Altitude','Bearing','CreatedOn','Latitude','Longitude','Provider','Speed','Time']
            separator => ','
            convert => {'Accuracy' => 'float','id'=>'integer'}
        }
    }

    output {
        elasticsearch {
                hosts => 'elasticsearch:9200'
        }
        stdout{

        }
    }

错误:

{:timestamp=>"2016-04-27T18:49:51.881000+0000", 
:message=>"fetched an invalid config", 
:config=>"input {\n\ttcp {\n\t\tport => 5000\n\t}\n\tfile {\n 
path => '/tmp/logstash/databases/1.csv'\n          
type => 'test'\n          start_position => 'beginning'\n 
}\n}\n\nfilter {\n    csv {\n        
columns => ['Id','ElapsedRealTimeNanos','Accuracy','Altitude','Bearing','CreatedOn','Latitude','Longitude','Provider','Speed','Time']\n        
separator => ','\n\t\t\t\tconvert => {'Accuracy' => 'float','id'=>'integer'}\n    }\n}\n\noutput {\n\telasticsearch {\n\t\t\thosts => 'elasticsearch:9200'\n\t}\n\tstdout{\n\n\t}\n}\n\n", 
:reason=>"Expected one of #, {, } at line 16, column 38 (byte 391) after filter {\n    csv {\n       
columns => ['Id','ElapsedRealTimeNanos','Accuracy','Altitude','Bearing','CreatedOn','Latitude','Longitude','Provider','Speed','Time']\n        
separator => ','\n\t\t\t\tconvert => {'Accuracy' => 'float'", :level=>:error}

1 个答案:

答案 0 :(得分:-1)

您可以将其用作:

Map distanceLabels = package1.distanceLabels;
for (Map.Entry<String, Integer> entry : distanceLabels.entrySet()) {
    String key = entry.getKey();
    Object value = entry.getValue();
    // ...
}

转化之间没有逗号。

您还可以使用---- configtest选项检查配置文件是否有错误:

...
convert => {
    'Accuracy' => 'float'
    'Id' => 'integer'
}
...