如何根据单列条件合并Logstash配置中的多个jdbc输入?

时间:2018-12-19 06:53:25

标签: logstash logstash-configuration logstash-jdbc

我在sql server中有两个表,即AppDetails和AppBranchDetails。 我想读取这两个表的所有行并根据条件进行合并。

下面是我要运行的两个查询:

  1. 按照ID从AppDetails顺序中选择id作为colg_id,名称,先生姓名
  2. 从AppBranchDetails中按ID依次选择id作为branch_id,branch_name,branch_add

在以上两个查询中,“ id”是主键,两个表都相同。

id == 1 的输出如下所示:

{
  "name": "ram",
  "sirname": "patil",
  "id": 1,
  "BRANCH": [
    {
      "id": 1,
      "branch_name": "EE",
      "branch_add": "IND"
    },
    {
      "id": 1,
      "branch_name": "ME",
      "branch_add": "IND"
    }
  ]
}

对于 id == 2 ,输出如下所示:

{
  "name": "sham",
  "sirname": "bhosle",
  "id": 2,
  "BRANCH": [
    {
      "id": 2,
      "branch_name": "SE",
      "branch_add": "US"
    },
    {
      "id": 2,
      "branch_name": "FE",
      "branch_add": "US"
    }
  ]
}

我正在尝试使用以下配置(app.conf):

input {
  jdbc {
    jdbc_connection_string => "jdbc:sqlserver://x.x.x.x:1433;databaseName=AAA;"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_user => "sa"
    jdbc_password => "sa@111"
    statement => "select id as colg_id, name, sirname from AppDetails order by id"
    tracking_column => "colg_id"
    use_column_value => true
    type => "college"
  }
  jdbc {
    jdbc_connection_string => "jdbc:sqlserver://x.x.x.x:1433;databaseName=AAA;"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_user => "sa"
    jdbc_password => "sa@111"
    statement => "select id as branch_id, branch_name, branch_add from AppBranchDetails order by id"
    tracking_column => "branch_id"
    use_column_value => true
    type => "branch"
  }
}

filter {
    if [type] == "branch" {
        aggregate {
        task_id => "%{branch_id}"
        code => "
            map['BRANCH'] ||= []
            map['BRANCH'] << event.get('branch_id')
            map['BRANCH'] << event.get('branch_name')
            map['BRANCH'] << event.get('branch_add')
            event.cancel()
        "
        push_previous_map_as_event => true
        timeout => 5
        }
        mutate {
            remove_field => [ "@version" , "@timestamp" ]
        }
    }
}

output {
  stdout { codec => json_lines }
}

任何人都可以建议我如何获得上面提到的结果。

0 个答案:

没有答案