Question

我想从oracle导入数据，并希望将导入数据的一个参数传递给弹性搜索以获取其他一些细节。

例如： - 如果我有一个员工ID，我从oracle db获得100行，我想将所有这100个员工ID传递给弹性搜索并获取emp名称和工资。

我现在可以从oracle检索数据，但无法连接到弹性搜索。此外，我不确定这样做会有什么更好的方法我正在使用log stash 2.3.3和弹性搜索日志存储过滤器插件。

 input {
       jdbc {
        jdbc_connection_string => "jdbc:oracle:thin:@<dbhost>:<port>:<sid>"
        # The user we wish to execute our statement as
        jdbc_user => “user"
        jdbc_password => “pass"
        # The path to our downloaded jdbc driver
        jdbc_driver_library => “<path>"
        # The name of the driver class for oracle
        jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
        # our query
        statement => "SELECT empId, desg from Employee"
    }
    elasticsearch {
      hosts => "https://xx.corp.com:9200"
        index => “empdetails”
    }
   }
   output {
    stdout { codec => json_lines }
   }

由于弹性搜索，我收到以下错误。

插件出现无法恢复的错误。将重新启动此插件。

插件：[＆＃34; https://xx.corp.com:9200＆＃34;]，index =＆gt;＆＃34; empdetails＆＃34;，query =＆gt;“empId：'1001＆＃39;＆＃34 ;，codec =＆gt;＆＃34; UTF-8＆＃34;＆gt;，scan =＆gt; true，size =＆gt; 1000，scroll =＆gt;＆＃34; 1m＆＃34;，docinfo =＆gt; false， docinfo_target =＆gt;＆＃34; @ metadata＆＃34;，docinfo_fields =＆gt; [＆＃34; _index＆＃34;，＆＃34; _type＆＃34;，＆＃34; _id＆＃34;]，ssl =＆gt ;假＆gt;

错误：[401] {：level =＆gt;：error}

Answer 1

您需要使用elasticsearch filter而不是elasticsearch input

input {
   jdbc {
    jdbc_connection_string => "jdbc:oracle:thin:@<dbhost>:<port>:<sid>"
    # The user we wish to execute our statement as
    jdbc_user => “user"
    jdbc_password => “pass"
    # The path to our downloaded jdbc driver
    jdbc_driver_library => “<path>"
    # The name of the driver class for oracle
    jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
    # our query
    statement => "SELECT empId, desg from Employee"
   }
}
filter {
  elasticsearch {
    hosts => ["xx.corp.com:9200"]
    query => "empId:%{empId}"
    user => "admin"
    password => "admin"
    sort => "empName:desc"
    fields => {
      "empName" => "empName" 
      "salary" => "salary" 
    }
  }
}
output {
  stdout { codec => json_lines }
}

因此，通过JDBC获取的每条记录都将通过ES中的相应数据进行丰富。

从输入中的logstash连接到弹性搜索的问题

1 个答案: