我尝试运行一个conf文件在输入和输出上使用mysql数据库我想得到一个json-line。但是错误正在发生。
我将错误和配置文件放在这个问题中。
Sending Logstash's logs to C:/logstash-6.0.1/logs which is now configured via log4j2.properties
[2017-12-13T17:33:43,978][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"C:/logstash-6.0.1/modules/fb_apache/configuration"}
[2017-12-13T17:33:43,978][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"fb_apache", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x1896b5d @module_name="fb_apache", @directory="C:/logstash-6.0.1/modules/fb_apache/configuration", @kibana_version_parts=["6", "0", "0"]>}
[2017-12-13T17:33:43,978][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"C:/logstash-6.0.1/modules/netflow/configuration"}
[2017-12-13T17:33:43,978][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"netflow", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x18b75ca @module_name="netflow", @directory="C:/logstash-6.0.1/modules/netflow/configuration", @kibana_version_parts=["6", "0", "0"]>}
[2017-12-13T17:33:44,038][DEBUG][logstash.config.source.multilocal] Reading pipeline configurations from YAML {:location=>"C:/logstash-6.0.1/config/pipelines.yml"}
[2017-12-13T17:33:44,038][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2017-12-13T17:33:44,068][DEBUG][logstash.agent ] Agent: Configuring metric collection
[2017-12-13T17:33:44,068][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-13T17:33:44,088][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-13T17:33:44,138][DEBUG][logstash.instrument.periodicpoller.persistentqueue] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-13T17:33:44,148][DEBUG][logstash.instrument.periodicpoller.deadletterqueue] PeriodicPoller: Starting {:polling_interval=>5, :polling_timeout=>120}
[2017-12-13T17:33:44,148][DEBUG][logstash.agent ] starting agent
[2017-12-13T17:33:44,158][DEBUG][logstash.agent ] Starting puma
[2017-12-13T17:33:44,158][DEBUG][logstash.agent ] Trying to start WebServer {:port=>9600}
[2017-12-13T17:33:44,168][DEBUG [logstash.config.source.local.configpathloader] Skipping the following files while reading config since they don't match the specified glob pattern {:files=>["C:/logstash-6.0.1/bin/BCHARTS-MTGOXUSD.csv", "C:/logstash-6.0.1/bin/cpdump", "C:/logstash-6.0.1/bin/ingest-convert.sh", "C:/logstash-6.0.1/bin/logstash", "C:/logstash-6.0.1/bin/logstash-plugin", "C:/logstash-6.0.1/bin/logstash-plugin.bat", "C:/logstash-6.0.1/bin/logstash.bat", "C:/logstash-6.0.1/bin/logstash.conf", "C:/logstash-6.0.1/bin/logstash.lib.sh", "C:/logstash-6.0.1/bin/mysql-connector-java-5.1.38.jar", "C:/logstash-6.0.1/bin/mysql-connector-java-5.1.45-bin.jar", "C:/logstash-6.0.1/bin/ruby", "C:/logstash-6.0.1/bin/setup.bat", "C:/logstash-6.0.1/bin/system-install"]}
[2017-12-13T17:33:44,178][DEBUG][logstash.api.service ] [api-service] start
[2017-12-13T17:33:44,178][DEBUG][logstash.config.source.local.configpathloader] Reading config file {:config_file=>"C:/logstash-6.0.1/bin/db.conf"}
[2017-12-13T17:33:44,208][DEBUG][logstash.agent ] Converging pipelines
[2017-12-13T17:33:44,208][DEBUG][logstash.agent ] Needed actions to converge {:actions_count=>1}
[2017-12-13T17:33:44,218][DEBUG][logstash.agent ] Executing action {:action=>LogStash::PipelineAction::Create/pipeline_id:main}
[2017-12-13T17:33:44,298][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2017-12-13T17:33:44,768][DEBUG][logstash.plugins.registry] On demand adding plugin to the registry {:name=>"jdbc", :type=>"input", :class=>LogStash::Inputs::Jdbc}
[2017-12-13T17:33:44,778][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"ArgumentError", :message=>"invalid byte sequence in UTF-8", :backtrace=>["org/jruby/RubyString.java:2541:in `gsub'", "C:/logstash-6.0.1/logstash-core/lib/logstash/util/environment_variables.rb:28:in `replace_env_placeholders'", "C:/logstash-6.0.1/logstash-core/lib/logstash/util/environment_variables.rb:18:in `deep_replace'", "C:/logstash-6.0.1/logstash-core/lib/logstash/config/mixin.rb:109:in `block in config_init'", "org/jruby/RubyHash.java:1343:in `each'", "C:/logstash-6.0.1/logstash-core/lib/logstash/config/mixin.rb:108:in `config_init'", "C:/logstash-6.0.1/logstash-core/lib/logstash/inputs/base.rb:62:in `initialize'", "C:/logstash-6.0.1/logstash-core/lib/logstash/pipeline.rb:152:in `plugin'", "(eval):8:in `<eval>'", "org/jruby/RubyKernel.java:994:in `eval'", "C:/logstash-6.0.1/logstash-core/lib/logstash/pipeline.rb:82:in `initialize'", "C:/logstash-6.0.1/logstash-core/lib/logstash/pipeline.rb:215:in `initialize'", "C:/logstash-6.0.1/logstash-core/lib/logstash/pipeline_action/create.rb:35:in `execute'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:335:in `block in converge_state'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:332:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:319:in `converge_state'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "C:/logstash-6.0.1/logstash-core/lib/logstash/agent.rb:90:in `execute'", "C:/logstash-6.0.1/logstash-core/lib/logstash/runner.rb:362:in `block in execute'", "C:/logstash-6.0.1/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
[2017-12-13T17:33:44,798][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Stopping
[2017-12-13T17:33:44,798][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Stopping
[2017-12-13T17:33:44,798][DEBUG][logstash.instrument.periodicpoller.persistentqueue] PeriodicPoller: Stopping
配置文件
input {
jdbc {
jdbc_driver_library => "C:\logstash-6.0.1\bin\mysql-connector-java-5.1.45-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/database"
jdbc_user => "root"
jdbc_password => "root"
statement => "SELECT * FROM table"
}
}
output {
stdout { codec => rubydebug }
}
修改
这是数据的第一行
cod,cod_veiculo.cod_destino,cod_origem
5530041,16555,84661,1187