Question

我有以下logstash配置文件：

input {
  jdbc {
    jdbc_driver_library => "C:\Program Files\Microsoft JDBC DRIVER 6.2 for SQL Server\sqljdbc_6.2\enu\mssql-jdbc-6.2.1.jre8"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_connection_string => "jdbc:sqlserver://TST-DBS-20;user=Elasticsearch;password=elastic123;"
    jdbc_user => "Elasticsearch"
    statement => "SELECT NewsID, HeadLine, BodyText, DateSubmitted, Approved, NULLIF(UserName,'') as UserName, NULLIF(Type,'') as NewsType, NULLIF(Caption,'') as caption, NULLIF(Author,'') as Author, NULLIF(Contact,'') as Contact, NULLIF(StaffID,'') as StaffID, SocialClubRegionID, DateCreated, CreatedBy, LastModifiedDate, ModifiedBy
FROM [News].[dbo].[News]"
  }
}
filter {
}
output {
  elasticsearch {
    hosts => ["tst-sch-20:9200"]
    index => "newsindex"
    document_id => "%{id}"
    user => "elastic"
    password => elastic123
  }
  stdout { codec => json }
}

我创建了以下索引：

put newsindex
{
  "settings" : {
    "number_of_shards":3,
    "number_of_replicas":2
  },
  "mappings" : {
    "news": {
      "properties": {
        "NewsId": {
          "type": "integer"
        },
        "newstype": {
          "type": "text"
        },
        "bodytext": {
          "type": "text"
        }
      }
    }
  }
}

运行上述脚本后，logstash日志文件中没有条目表明出现任何问题。如果我直接在SQL中运行SQL命令，那么奇怪的是，索引中的单个条目是我的select语句的最后一个条目，所以它几乎就像脚本插入然后覆盖一样，以便我最终得到一条记录。

Answer 1

如果您查看加载到Elasticsearch中的记录的_id字段，您会看到它是%{id}，因为您的查询没有id字段。您需要根据查询更改为document_id => "%{newsid}"或任何有意义的内容。

logstash只在我的索引中放置一个条目

1 个答案: