How to use custom avro schema with dockerized connect-datagen?

时间:2019-04-17 02:26:52

标签: apache-kafka avro apache-kafka-connect confluent confluent-schema-registry

I'm trying to generate test data from custom avro schema's using dockerized connect-datagen.

Here is an example on Github: https://github.com/damc-dev/kafka-docker-datagen

I mapped the directory to the connect container and configured to use a custom schema based on the instructions here: https://github.com/confluentinc/kafka-connect-datagen/blob/master/README.md#define-a-new-schema-specification

I create a topic and create the connector, but it doesn't generate data on the topic and the error message in the connect logs is:

connect            | [2019-04-17 02:19:38,317] ERROR WorkerSourceTask{id=datagen-impressions-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
connect            | java.lang.NullPointerException
connect            |    at org.apache.avro.Schema.parse(Schema.java:1225)
connect            |    at org.apache.avro.Schema$Parser.parse(Schema.java:1032)
connect            |    at org.apache.avro.Schema$Parser.parse(Schema.java:1004)
connect            |    at io.confluent.avro.random.generator.Generator.<init>(Generator.java:218)
connect            |    at io.confluent.kafka.connect.datagen.DatagenTask.start(DatagenTask.java:120)
connect            |    at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:199)
connect            |    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
connect            |    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
connect            |    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
connect            |    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
connect            |    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
connect            |    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
connect            |    at java.lang.Thread.run(Thread.java:748)
connect            | [2019-04-17 02:19:38,319] ERROR WorkerSourceTask{id=datagen-impressions-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)

0 个答案:

没有答案