在elasticsearch和Kibana中导入.log文件

Question

我有这个带有JSON的.log文件，看起来像这样

    {"method":"GET","path":"/public/index","format":"html","controller":"public","action":"index","status":200,"duration":84.59,"view":33.7,"db":47.45,"ip":"127.0.0.1","route":"public#index","request_id":"4d7016832294bafa8f593453eed2adb1","source":"unknown","tags":["request"],"@timestamp":"2018-11-09T22:54:06Z","@version":"1"}
{"method":"GET","path":"/public/index","format":"html","controller":"public","action":"index","status":200,"duration":15.44,"view":13.85,"db":0.91,"ip":null,"route":null,"request_id":null,"source":"unknown","tags":["request"],"@timestamp":"2018-11-09T22:54:28Z","@version":"1"}
{"method":"GET","path":"/public/index","format":"html","controller":"public","action":"index","status":200,"duration":13.86,"view":12.47,"db":0.8,"ip":null,"route":null,"request_id":null,"source":"unknown","tags":["request"],"@timestamp":"2018-11-09T22:54:40Z","@version":"1"}

，我尝试将其导入elasticsearch和Kibana中。 我在命令行

中尝试了此代码

curl -XPOST "http://localhost:9200/test/test" -H "Content-Type: application/json" -d @logfile.log

我收到此错误

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"illegal_argument_exception","reason":"Malformed content, found extra data after parsing: START_OBJECT"}},"status":400}

当我看着Kibana时，我看到了

我做错了什么？

Answer 1

批量格式为

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....

文件中已经包含所有optional_source行。

只需添加包含以下内容的行 { "index" : { "_index" : "YOUR-INDEX-NAME", "_type" : "_doc"} } 每行之前。

然后针对ES https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html中的批量api进行POST