将数据从mysql迁移到AWS Elasticsearch Logstash

时间:2018-11-19 09:55:31

标签: mysql elasticsearch amazon-s3 plugins logstash

我是ELK的新手,以前从未使用过AWS Elasticsearch。

我正在尝试使用Logstash将数据从mysql服务器迁移到AWS Elasticsearch。

但是,我遇到这样的错误:

[ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::OutputDelegator:0x3ad80dbe>", :error=>"undefined method `credentials' for nil:NilClass", :thread=>"#<Thread:0x1cddccdd run>"}
[ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<NoMethodError: undefined method `credentials' for nil:NilClass>,
[ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Stop/pipeline_id:main, :exception=>"NoMethodError", :message=>"undefined method `unlock' for nil:NilClass"

我使用以下命令安装了aws logstash插件:

bin/logstash-plugin install logstash-output-amazon_es

我的配置文件如下:

input {
  jdbc {
    jdbc_driver_library => "/library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home/lib/mysql-connector-java-8.0.13.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql:SOMETHING"
    jdbc_user => "USER"
    jdbc_password => "PASSWORD"
    statement => "select * from TABLE_NAME"

      }
    }

output {
  amazon_es {
    hosts => ["MY_AWS_ES_DOMAIN"]
    region => "REGION"
    index => "INDEX_NAME"
    document_type => "TYPE"
    }
  stdout {
    codec => rubydebug
  }
}

请帮助我找到解决方法:'(( 提前非常感谢您!

谢谢, 山姆

0 个答案:

没有答案