Question

我想将json文件数据导入弹性搜索。这是我的logstash配置文件 -

输入{file {       type =＆gt; “JSON”       path =＆gt; “C：\用户\桌面\ newJSON.json”       start_position =＆gt; “开始”sincedb_path =＆gt; “\ dev的\空”                         }}

输出{       stdout {           codec =＆gt; rubydebug       }       elasticsearch {           hosts =＆gt; “本地主机：9200”           index =＆gt; “jsondata1”       }}

这是我的json文件---

{ “水果”：“苹果”， “大小”：“小”， “红色” }， { “水果”：“木瓜”， “大小”：“大”， “颜色”：“黄色” “测试”：“甜蜜” }

我使用此命令执行上面的配置文件----

logstash -f logstashcon.conf

但我在弹性搜索索引中得到了如下数据 -

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 10,
    "max_score": 1,
    "hits": [
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablA",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.302Z",
          "@version": "1",
          "host": "user-102",
          "message": "{\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablB",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.694Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"fruit\": \"Apple\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablE",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.696Z",
          "@version": "1",
          "host": "user-102",
          "message": "},\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablC",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.695Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"size\": \"Large\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablD",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.696Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"color\": \"Red\"\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablG",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.698Z",
          "@version": "1",
          "host": "user-102",
          "message": "\"fruit\": \"Papaya\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablJ",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "}\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablH",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"size\": \"Large\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablF",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.698Z",
          "@version": "1",
          "host": "user-102",
          "message": "{\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablI",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"color\": \"Yellow\"\r",
          "type": "json"
        }
      }
    ]
  }
}

请帮助我获得正确的输出谢谢！

Answer 1

您需要使用logstash的JSON过滤器插件来构建输出，

例如，如果您在message字段中有JSON数据，那么您的过滤器就会是，

filter {
  json {
    source => "message"
  }
}

这是文档所说的，

它需要一个包含JSON的现有字段并将其扩展为 Logstash事件中的实际数据结构。

请详细了解usage and example here。

使用logstash将json文件数据导入弹性搜索

1 个答案: