如何将连接器安装到Apache Kafka Connect的Docker映像

时间:2020-07-30 11:44:28

标签: docker apache-kafka apache-kafka-connect

我正在使用https://hub.docker.com/r/1ambda/kafka-connect/在ubuntu上安装docker kafka connect。我可以运行它,但无法在其中安装更多连接器。

我尝试过的

1-我尝试将连接器文件从我的机器复制到docker container连接器文件夹并重新启动,但是api,http:// localhost:8080 / connectors给出了空数组。

2-还在connect-distributed.properties中添加了插件路径。 怎么做这个主意??

2 个答案:

答案 0 :(得分:0)

1-我尝试将连接器文件从我的机器复制到docker container连接器文件夹并重新启动,但是api,http:// localhost:8080 / connectors给出了空数组。

localhost:8083 / connectors,将为您提供活动连接器的列表。 如果您需要检查可用插件的列表,则应点击localhost:8083 / connector-plugins

curl localhost:8083/connector-plugins
[{"class":"io.confluent.connect.activemq.ActiveMQSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.elasticsearch.ElasticsearchSinkConnector","type":"sink","version":"5.5.1"},{"class":"io.confluent.connect.ibm.mq.IbmMQSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.jdbc.JdbcSinkConnector","type":"sink","version":"5.5.1"},
{"class":"io.confluent.connect.jdbc.JdbcSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.jms.JmsSourceConnector","type":"source","version":"5.5.1"},{"class":"io.confluent.connect.s3.S3SinkConnector","type":"sink","version":"5.5.1"},{"class":"io.confluent.connect.storage.tools.SchemaSourceConnector","type":"source","version":"5.5.1-ccs"},{"class":"io.confluent.kafka.connect.datagen.DatagenConnector","type":"source","version":"null"},{"class":"org.apache.kafka.connect.file.FileStreamSinkConnector","type":"sink","version":"5.5.1-ccs"}, 
{"class":"org.apache.kafka.connect.file.FileStreamSourceConnector","type":"source","version":"5.5.1-ccs"},{"class":"org.apache.kafka.connect.mirror.MirrorCheckpointConnector","type":"source","version":"1"},{"class":"org.apache.kafka.connect.mirror.MirrorHeartbeatConnector","type":"source","version":"1"},{"class":"org.apache.kafka.connect.mirror.MirrorSourceConnector","type":"source","version":"1"}]

此外,需要将新的连接器添加到您所有工作节点的插件路径中,并且所有连接都需要重新启动

要进行进一步的故障排除,您可以尝试将日志级别提高到DEBUG,并检查启动日志(如果正在遍历连接器路径) 您将看到以下格式的日志:

DEBUG Loading plugin urls: ......

希望这有助于解决您的问题:)

答案 1 :(得分:0)

由于使用Kafka 0.10,该图像似乎未更新为可以与TableDestination:=ws.Cells(1,1) 一起使用

您可以使用plugin.path安装连接器,如我在Connect容器中记录的那样

https://github.com/OneCricketeer/apache-kafka-connect-docker