融合BigQuerySinkConnector JSON转换器错误

时间:2020-08-19 17:12:55

标签: google-bigquery apache-kafka-connect confluent-platform

我尝试使用有关Kafka主题的JSON数据创建到BigQuery的数据管道。测试环境是一个cp-all-in-one docker容器。我添加了它(默认情况下不存在)。

可以通过REST API毫无问题地发送测试生产者数据

{“ records”:[{“ key”:{“ id”:“ bar”},“ value”:{“ foo”:“ bar”}}]}

enter image description here

enter image description here

我创建了如下连接器配置文件:

# A tibble: 4 x 3
  size  anim      y
  <chr> <chr> <dbl>
1 big   Ape       1
2 small Ape       2
3 big   Dog       5
4 small Dog       3

我根据Robin Moffat的suggestions

设置了转换器程序包

当我使用此配置文件创建连接器时,连接器会出现如下错误:

enter image description here

{
   "name": "kcbq-connect1",
   "config": {
     "connector.class": "com.wepay.kafka.connect.bigquery.BigQuerySinkConnector",
     "tasks.max" : "1",
     "topics" : "jsontest2",
     "sanitizeTopics" : "true",
     "sanitizeFieldNames" : "true",
     "autoCreateTables" : "true",
     "autoUpdateSchemas" : "true",
     "schemaRetriever" : "com.wepay.kafka.connect.bigquery.schemaregistry.schemaretriever.SchemaRegistrySchemaRetriever",
     "schemaRegistryLocation":"http://localhost:8081",
     "bufferSize": "100000",
     "maxWriteSize":"10000",
     "tableWriteWait": "1000",
     "project" : "dream-puzzle-6bf2f",
     "datasets" : ".*=onurtest",
     "keyfile" : " /tmp/dream-puzzle-6bf2f-d3e4e38d8dbd.json",
     "value.converter" : "org.apache.kafka.connect.json.JsonConverter",
     "value.converter.schemas.enable" : "false",
     "key.converter" : "org.apache.kafka.connect.json.JsonConverter",
     "key.converter.schemas.enable" : "false"
   }
 }

此NullPointerException来自何处?

0 个答案:

没有答案