Question

我正在使用Logstash 2.3.3将数据从csv文件加载到Elasticsearch 2.3.3。

这是我的Logstash配置文件：

input {
  #stdin {
   # type => "stdin-type"
  #}

  file {
    path => "/Users/hieupm/export.txt"
    start_position => "beginning"
    codec => plain {
      charset => "CP1252"
    }
  }
}

filter {
  mutate {
    gsub => [
      "message", "\"", " ",
      "message", " +", " ",
      "message", " ;", ";",
      "message", "; ", ";"
    ]
    lowercase => ["message"]
  }
  csv {
    columns => ["siren","raison_sociale","enseigne","cp_ville","naf","nature_juridique","code_tranche"] 
    separator => ","
  }
}

output {
  #stdout {
  #}

  elasticsearch {
    hosts => "127.0.0.1:9200"
    index => "ereputation"
    document_id => "%{siren}"
  }
}

执行日志（在终端中）没有给出任何错误，但它只显示如下并且从未完成

Settings: Default pipeline workers: 8
Pipeline main started

我的示例数据：

siren,raison_sociale,enseigne,cp_ville,naf,nature_juridique,code_tranche 
5420021,ETABLISSEMENTS BIQUEZ,,80100 ABBEVILLE,4669B,5710,11
5420120,STE DES SUCRERIES DU MARQUENTERRE,,62140 MARCONNELLE,1081Z,5599,11
5480082,STE BRETONNE DE TRAVAUX PUBLICS,,44600 ST NAZAIRE,4312A,5599,21
5480546,SOCIETE DE DISTRIBUTION DE PAPIER,,44350 GUERANDE,1712Z,5710,21
5520176,CARTONNAGE HERNAS,,80210 FEUQUIERES EN VIMEU,1721A,5710,21
5520242,PROSPA,,80510 LONGPRE LES CORPS SAINTS,2030Z,5710,12
5520325,ETABLISSEMENTS ADRIEN RIQUIER,,80570 DARGNIES,4674B,5710,12
5540380,SNC COMTE ET CIE,PHARMACIE COMTE,04000 DIGNE LES BAINS,4773Z,5202,11
5541552,ETABLISSEMENTS BONNET FRERES,,04370 BEAUVEZER,5610C,5499,03

感谢您的任何建议！

Answer 1

我是elasticsearch_loader的作者如果您只想将CSV数据加载到elasticsearch中，可以使用elasticsearch_loader
我为这个确切的问题编写了ESL

您可以使用pip下载：

elasticsearch_loader --index ereputation --type incident csv input1.csv input2.csv

然后你可以通过发出以下命令将csv / json / parquet文件加载到elasticsearch：

currying

无法使用Logstash将数据从CSV加载到Elasticsearch

1 个答案: