Question

我正在执行一个Logstash管道，该管道通过Simba JDBC驱动程序查询BigQuery：

input {
    jdbc {
        id => "categories-input"
        jdbc_user => "user"
        jdbc_driver_library => "..."
        jdbc_driver_class => "com.simba.googlebigquery.jdbc42.Driver"
        jdbc_connection_string => "..."
        statement => "
            SELECT categoryid, categoryname, filters
            FROM dataset.categories
            WHERE categoryid = 12345;"
    }
}

filter {
    mutate {
        id => "categories-mapping"
        rename => {
            "categoryid" => "[category][id]"
            "categoryname" => "[category][name]"
        }
        remove_field => ["@timestamp", "@version"]
    }
}

output {
    elasticsearch {
        template => "/usr/share/logstash/shared/generic.json"
        template_name => "generic"
        id => "categories-updating"
        hosts => "elasticsearch:9200"
        index => "categories"
        document_id => "%{[category][id]}"
    }
}

此外，我的整个dataset.categories表很小，不到100兆字节

但是我的filters字符串很长（准确地说是10,796,031个字符），结果出现以下异常：

即使我不将过滤器转换为字符串，我也会遇到同样的异常。

执行JDBC查询时发生异常 {：exception =>＃BigQueryJDBCDriver从以下位置获取结果时出错缓冲。>}

如果有帮助，那就是我的连接字符串：

jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=VerySeriousProject;OAuthType=0;OAuthServiceAcctEmail=veryimportantemail@VerySeriousProject.iam.gserviceaccount.com;OAuthPvtKeyPath=/usr/share/logstash/drivers/key.json;Timeout=600;AllowLargeResults=1;

为了不接收此类异常并能够查询数据，我需要更改什么？

从缓冲区获取结果时出错。 Simba JDBC BigQuery驱动程序

0 个答案: