Logstash打破了最近对JDBC mySQL连接器的重命名

时间：2018-08-31 14:17:36

标签： mysql jdbc logstash connector

由于某种原因，带有弹性堆栈X-Pack的Logstash损坏。我认为这与最近对mySQL连接器进行的重命名有关，该名称在各种配置文件中均未更新。但是，在此错误日志中找不到错误文件的来源。另外，如果有人知道如何将实际的mySQL连接器从com.mysql.cj.jdbc.Driver重命名为com.mysql.jdbc.Driver，这应该可以解决所有问题。

错误日志：

C:\Program Files\logstash-6.3.2\bin>logstash -f sql.conf
Sending Logstash's logs to C:/Program Files/logstash-6.3.2/logs which is now configured via log4j2.properties
[2018-08-31T15:01:52,502][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-08-31T15:01:53,031][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.3.2"}
[2018-08-31T15:01:55,217][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-08-31T15:02:06,342][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-08-31T15:02:06,342][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-08-31T15:02:06,539][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-08-31T15:02:06,575][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-08-31T15:02:06,592][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-08-31T15:02:06,609][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-08-31T15:02:06,625][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-08-31T15:02:06,659][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-08-31T15:02:06,909][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x598b8674 sleep>"}
[2018-08-31T15:02:06,996][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-08-31T15:02:07,359][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-08-31T15:02:07,895][ERROR][logstash.inputs.jdbc     ] Unable to connect to database. Tried 1 times {:error_message=>"Java::JavaSql::SQLNonTransientConnectionException: Cannot load connection class because of underlying exception: com.mysql.cj.exceptions.WrongArgumentException: Failed to parse the host:port pair 'localhost:3306;user=test;password=test123;databaseName=test;integratedSecurity=true;'."}
[2018-08-31T15:02:07,916][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Jdbc jdbc_connection_string=>"jdbc:mysql://localhost:3306;user=test;password=test123;databaseName=test;integratedSecurity=true;", jdbc_driver_class=>"com.mysql.cj.jdbc.Driver", jdbc_user=>"doesntmatterwithauthentication", statement=>"SELECT * FROM phones", id=>"de31f73d4505e1de7e76bce4917c48b412909473f3872288edd51acccf0e0be6", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_ad3ad9d5-a3e8-492a-9231-1e729e8c4190", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, parameters=>{"sql_last_value"=>1970-01-01 01:00:00 +0100}, last_run_metadata_path=>"C:\\Users\\ross.massie/.logstash_jdbc_last_run", use_column_value=>false, tracking_column_type=>"numeric", clean_run=>false, record_last_run=>true, lowercase_column_names=>true>
  Error: Java::JavaSql::SQLNonTransientConnectionException: Cannot load connection class because of underlying exception: com.mysql.cj.exceptions.WrongArgumentException: Failed to parse the host:port pair 'localhost:3306;user=test;password=test123;databaseName=test;integratedSecurity=true;'.
  Exception: Sequel::DatabaseConnectionError
  Stack: com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(com/mysql/cj/jdbc/exceptions/SQLError.java:110)
com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(com/mysql/cj/jdbc/exceptions/SQLError.java:97)
com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(com/mysql/cj/jdbc/exceptions/SQLError.java:89)
com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(com/mysql/cj/jdbc/exceptions/SQLError.java:63)
com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(com/mysql/cj/jdbc/exceptions/SQLError.java:73)
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(com/mysql/cj/jdbc/exceptions/SQLExceptionsMapping.java:79)
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(com/mysql/cj/jdbc/exceptions/SQLExceptionsMapping.java:131)
com.mysql.cj.jdbc.NonRegisteringDriver.connect(com/mysql/cj/jdbc/NonRegisteringDriver.java:227)
java.lang.reflect.Method.invoke(java/lang/reflect/Method)
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:423)
org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:290)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.adapters.jdbc.connect(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/adapters/jdbc.rb:215)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.adapters.jdbc.RUBY$method$connect$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/sequel_minus_5_dot_10_dot_0/lib/sequel/adapters/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/adapters/jdbc.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.connection_pool.make_new(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/connection_pool.rb:127)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.connection_pool.RUBY$method$make_new$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/sequel_minus_5_dot_10_dot_0/lib/sequel/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/connection_pool.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.connection_pool.threaded.assign_connection(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/connection_pool/threaded.rb:206)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.connection_pool.threaded.RUBY$method$assign_connection$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/sequel_minus_5_dot_10_dot_0/lib/sequel/connection_pool/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/connection_pool/threaded.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.connection_pool.threaded.acquire(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/connection_pool/threaded.rb:138)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.connection_pool.threaded.RUBY$method$acquire$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/sequel_minus_5_dot_10_dot_0/lib/sequel/connection_pool/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/connection_pool/threaded.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.connection_pool.threaded.hold(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/connection_pool/threaded.rb:90)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.database.connecting.synchronize(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/database/connecting.rb:270)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.database.connecting.test_connection(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/database/connecting.rb:279)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.database.connecting.connect(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/database/connecting.rb:58)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.sequel_minus_5_dot_10_dot_0.lib.sequel.core.connect(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/sequel-5.10.0/lib/sequel/core.rb:116)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.plugin_mixins.jdbc.block in jdbc_connect(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb:114)
org.jruby.RubyKernel.loop(org/jruby/RubyKernel.java:1292)
org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(org/jruby/RubyKernel$INVOKER$s$0$0$loop.gen)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.plugin_mixins.jdbc.jdbc_connect(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb:111)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.plugin_mixins.jdbc.RUBY$method$jdbc_connect$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9/lib/logstash/plugin_mixins/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.plugin_mixins.jdbc.open_jdbc_connection(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb:164)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.plugin_mixins.jdbc.RUBY$method$open_jdbc_connection$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9/lib/logstash/plugin_mixins/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.plugin_mixins.jdbc.execute_statement(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb:220)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.plugin_mixins.jdbc.RUBY$method$execute_statement$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9/lib/logstash/plugin_mixins/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/plugin_mixins/jdbc.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.inputs.jdbc.execute_query(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb:264)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.inputs.jdbc.RUBY$method$execute_query$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9/lib/logstash/inputs/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.inputs.jdbc.run(C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb:250)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9.lib.logstash.inputs.jdbc.RUBY$method$run$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_jdbc_minus_4_dot_3_dot_9/lib/logstash/inputs/C:/Program Files/logstash-6.3.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.9/lib/logstash/inputs/jdbc.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.logstash_minus_core.lib.logstash.pipeline.inputworker(C:/Program Files/logstash-6.3.2/logstash-core/lib/logstash/pipeline.rb:512)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.logstash_minus_core.lib.logstash.pipeline.RUBY$method$inputworker$0$__VARARGS__(C_3a_/Program_20_Files/logstash_minus_6_dot_3_dot_2/logstash_minus_core/lib/logstash/C:/Program Files/logstash-6.3.2/logstash-core/lib/logstash/pipeline.rb)
C_3a_.Program_20_Files.logstash_minus_6_dot_3_dot_2.logstash_minus_core.lib.logstash.pipeline.block in start_input(C:/Program Files/logstash-6.3.2/logstash-core/lib/logstash/pipeline.rb:505)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:289)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:246)

sql.conf：

input {
  jdbc {
    jdbc_connection_string => "jdbc:mysql://localhost:3306/test?useSSL=false&amp;serverTimezone=GMT&amp;DatabaseName=test"
    jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
    jdbc_driver_library => "C:\Program Files (x86)\MySQL\Connector J 8.0\mysql-connector-java-8.0.12.jar"
    jdbc_user => "test"
    jdbc_password => "test123"

    statement => "SELECT * FROM phones"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "phones"
  }
}

1 个答案:

答案 0 :(得分：0)

几次调整后，重写整个连接字符串即可。发行时，我发现x-pack中有一些驱动程序引用，如果内存可用并且需要更改，则引用了错误的驱动程序。

相关问题
最新问题