Question

我从csv解析日期时遇到问题，我无法找到问题（人们会假设）简单的日期 - dd / MM / yy。这是我的csv文件的结构：

Date,Key-values,Line Item,Creative,Ad unit,Creative size,Ad server impressions,Ad server clicks,Ad server CTR
04/04/16,prid=DUBAP,Hilton_PostAuth 1,Stop Clicking Around - 300x250,383UKHilton_300x250,300 x 250,31,0,0.00%
04/04/16,prid=DUBAP,Hilton_PostAuth 2,16-0006_Auction_Banners_300x250_cat4,383UKHilton_300x250,300 x 250,59,0,0.00%

和我的logstash.config文件：

input {
  file {
    path => "/Users/User/*.csv"
    type => "core2"
    start_position => "beginning"    
  }
}
filter {
  csv {
      columns => ["Date","Key-values","Line Item","Creative","Ad unit","Creative size","Ad server impressions","Ad server clicks","Ad server CTR"]
      separator => ","
  }
  date {
    match => ["Date", "dd/MM/YY"]
  }
  mutate {convert => ["Ad server impressions", "float"]}
  mutate {convert => ["Ad server clicks", "float"]}
  mutate {convert => ["Ad server CTR", "float"]}
}
output {  
    elasticsearch {
        action => "index"
        hosts => "localhost"
        index => "test1"
        workers => 1
    }
    stdout {}
}

我也尝试过将日期设为“dd / MM / yy”而没有运气的组合，日期没有作为日期编入索引，我只能在Kibana中选择@timestamp ..

我认为必须有一个简单的事情，我只是错过了，但至于这一刻我找不到它..

干杯！

编辑1：

请在logstash启动时找到我的控制台输出以及如何处理数据：

Settings: Default pipeline workers: 4
Pipeline main started
Failed parsing date from field {:field=>"Date", :value=>"Date", :exception=>"Invalid format: \"Date\"", :config_parsers=>"dd/MM/YY", :config_locale=>"default=en_US", :level=>:warn}
2016-05-06T20:32:48.034Z Pawels-MacBook-Air.local Date,Key-values,Line Item,Creative,Ad unit,Creative size,Ad server impressions,Ad server clicks,Ad server CTR
2016-04-03T23:00:00.000Z Pawels-MacBook-Air.local 04/04/16,prid=DUBAP,Hilton_PostAuth 1,Stop Clicking Around - 300x250,383UKHilton_300x250,300 x 250,31,0,0.00%

它仍然将它加载到Elasticsearch中，但在Kibana中没有'Date'字段 - 我只能使用@timestamp

干杯

Answer 1

实际上date过滤器的作用是：

日期过滤器用于解析字段中的日期，然后使用该日期或时间戳作为事件的logstash时间戳。

因此，使用该配置，它会读取您的日期并将其用作时间戳字段。如果要将其用作单独的字段，请配置为：

date {
    match => ["Date", "dd/MM/yy"]
    target => "Date"
}

Logstash解析csv日期

1 个答案: