Question

我正在尝试使用Logstash将数据从oracle加载到Elastic。

弹性7.0.1 Logstash 7.0.1

logstash.conf：

input {
    jdbc {
        # The path to our downloaded jdbc driver
        jdbc_driver_library => "/home/foobar/apps/elastic/ojdbc7.jar"
        # Postgres jdbc connection string to our database, mydb
        jdbc_connection_string => "jdbc:oracle:thin:@localhost:1521:xe"
        # The user we wish to execute our statement as
        jdbc_user => "scott"
        jdbc_password => "tiger"
        # The name of the driver class for Postgresql
        jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
        # our query
        statement => "select log_id,log_name,log_type from log_test where rownum < 1000"
    }
}
output {
    stdout { codec => json_lines }
    elasticsearch {
        "hosts" => "localhost:9200"
        "index" => "logtest"
        "document_type" => "data"
        "document_id" =>"%{log_id}"
    }
}

运行命令：

/home/foobar/apps/elastic/logstash-7.0.1/bin/logstash --path.settings /home/foobar/apps/elastic/logstash-7.0.1/ -f ./logstash-jdbc.conf

输出：

WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /home/foobar/apps/elastic/logstash-7.0.1/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2019-05-15 09:32:58.564 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2019-05-15 09:32:58.587 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"7.0.1"}
[ERROR] 2019-05-15 09:33:05.738 [Converge PipelineAction::Create<main>] elasticsearch - Unknown setting '"document_type"' for elasticsearch
[ERROR] 2019-05-15 09:33:05.738 [Converge PipelineAction::Create<main>] elasticsearch - Unknown setting '"hosts"' for elasticsearch
[ERROR] 2019-05-15 09:33:05.738 [Converge PipelineAction::Create<main>] elasticsearch - Unknown setting '"index"' for elasticsearch
[ERROR] 2019-05-15 09:33:05.750 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/home/foobar/apps/elastic/logstash-7.0.1/logstash-core/lib/logstash/config/mixin.rb:86:in `config_init'", "/home/foobar/apps/elastic/logstash-7.0.1/logstash-core/lib/logstash/outputs/base.rb:60:in `initialize'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:232:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:48:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:30:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:239:in `plugin'", "org/logstash/plugins/PluginFactoryExt.java:137:in `buildOutput'", "org/logstash/execution/JavaBasePipelineExt.java:50:in `initialize'", "/home/foobar/apps/elastic/logstash-7.0.1/logstash-core/lib/logstash/java_pipeline.rb:23:in `initialize'", "/home/foobar/apps/elastic/logstash-7.0.1/logstash-core/lib/logstash/pipeline_action/create.rb:36:in `execute'", "/home/foobar/apps/elastic/logstash-7.0.1/logstash-core/lib/logstash/agent.rb:325:in `block in converge_state'"]}
[INFO ] 2019-05-15 09:33:06.178 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2019-05-15 09:33:11.085 [LogStash::Runner] runner - Logstash shut down.

对错误更感兴趣：

[ERROR] elasticsearch - Unknown setting '"document_type"' for elasticsearch
[ERROR] elasticsearch - Unknown setting '"hosts"' for elasticsearch
[ERROR] elasticsearch - Unknown setting '"index"' for elasticsearch
[ERROR] elasticsearch - Unknown setting '"document_id"' for elasticsearch
[ERROR] agent - Failed to execute action

它与logstash 6.5一起使用，但是自从我升级到7.0.1以来不起作用。

任何帮助将不胜感激。

Answer 1

elasticsearch输出的配置选项没有双引号。

将输出更改为此：

output {
    stdout { codec => json_lines }
    elasticsearch {
        hosts => "localhost:9200"
        index => "logtest"
        document_type => "data"
        document_id =>"%{log_id}"
    }
}

document_type也已被弃用，您应该避免使用它。

Logstash错误-Elasticsearch的未知设置“主机”

1 个答案: