Question

I am trying to evaluate the kafka confluent platform version 4.0. However, when I enabled ssl on the kafka cluster, the kafka-connect is failed to start.

logging detailed as below:

[2017-12-18 04:38:55,747] ERROR Uncaught exception in herder work thread, exiting:  (org.apache.kafka.connect.runtime.distributed.DistributedHerder:218)
org.apache.kafka.connect.errors.ConnectException: Timed out while checking for or creating topic(s) 'connect-offsets'. This could indicate a connectivity issue, unavailable topic partitions, or if this is your first use of the topic it may have taken too long to create.
        at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:243)
        at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$1.run(KafkaOffsetBackingStore.java:99)
        at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:126)
        at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:109)
        at org.apache.kafka.connect.runtime.Worker.start(Worker.java:144)
        at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:100)
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:205)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
[2017-12-18 04:38:55,752] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:65)
[2017-12-18 04:38:55,753] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:154)
[2017-12-18 04:38:55,761] INFO Stopped ServerConnector@51012d9c{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:306)
[2017-12-18 04:38:55,783] INFO Stopped o.e.j.s.ServletContextHandler@77a001d8{/,null,UNAVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:865)
[2017-12-18 04:38:55,786] INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer:165)
[2017-12-18 04:38:55,787] INFO Herder stopping (org.apache.kafka.connect.runtime.distributed.DistributedHerder:389)
[2017-12-18 04:39:00,788] INFO Herder stopped (org.apache.kafka.connect.runtime.distributed.DistributedHerder:409)
[2017-12-18 04:39:00,789] INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect:70)

I have checked the kafka broker, they are still running as normal.

bin/confluent status
connect is [DOWN]
kafka-rest is [UP]
schema-registry is [DOWN]
kafka is [UP]
zookeeper is [UP]

Any extra configuration that I have missed?

Please advise?

Answer 1

Once you have enabled security options on your Kafka cluster you need to enable the equivalent options to your Kafka Connect workers.

For example, for basic SSL configuration you might need to set something like:

security.protocol=SSL
ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks
ssl.truststore.password=<your-pass>

to your worker's configuration. (To try with Confluent CLI as you show above, this file is ./etc/schema-registry/connect-avro-distributed.properties)

You may read more on how to setup Kafka Connect with security here:

https://docs.confluent.io/current/connect/security.html

and find out about all the available security-related properties for Connect workers here:

https://docs.confluent.io/current/connect/allconfigs.html

kafka-connect failed to start when enabling ssl on kafka cluster

1 个答案: