启动kafka-gcs融合连接器时,找不到io.confluent.connect.gcs.GcsSinkConnector类

时间:2018-10-30 16:21:52

标签: apache-kafka google-cloud-storage ubuntu-16.04 apache-kafka-connect confluent

我正在为kafka和Google Cloud Storage(GCS)使用融合连接器。我使用以下方法安装了汇合版:

(我正在Ubuntu 16.04上执行此操作,所有这些命令都在主目录中执行。)

curl https://packages.confluent.io/archive/5.0/confluent-5.0.0-2.11.tar.gz | tar xz

然后我通过以下方式安装了confluent-hub:

curl http://client.hub.confluent.io/confluent-hub-client-latest.tar.gz | tar xz

然后我使用以下命令安装kafka-GCS连接器:

confluent-hub install confluentinc/kafka-connect-gcs:5.0.1

然后我使用以下命令导出路径:

export PATH=<insert-path-here>/confluent-5.0.0/bin:$PATH

然后我使用以下方法开始融合:

confluent start

它显示:

This CLI is intended for development only, not for production
https://docs.confluent.io/current/cli/index.html

Using CONFLUENT_CURRENT: /tmp/confluent.pc9RaNNQ
Starting zookeeper
zookeeper is [UP]
Starting kafka
kafka is [UP]
Starting schema-registry
schema-registry is [UP]
Starting kafka-rest
kafka-rest is [UP]
Starting connect
connect is [UP]
Starting ksql-server
ksql-server is [UP]
Starting control-center
control-center is [UP]

通过以下方式检查融合物:

confluent status

给我:

This CLI is intended for development only, not for production
https://docs.confluent.io/current/cli/index.html

control-center is [UP]
ksql-server is [UP]
connect is [UP]
kafka-rest is [UP]
schema-registry is [UP]
kafka is [UP]
zookeeper is [UP]

到目前为止一切都很好。 Zookeeper和kafka都已启动并正在运行。现在,当我使用以下命令启动kafka-gcs融合连接器时:

confluent load gcs -d quickstart-gcs.properties

它出现以下错误:

Failed to find any class that implements Connector and which name 
matches io.confluent.connect.gcs.GcsSinkConnector

quickstart-gcs.properties文件位于主目录中,看起来像这样:

name=gcs-sink
connector.class=io.confluent.connect.gcs.GcsSinkConnector
tasks.max=1
topics=test

gcs.bucket.name=test_bucket
gcs.part.size=5242880
flush.size=3

gcs.credentials.path=/path/to/kafka/key.json

storage.class=io.confluent.connect.gcs.storage.GcsStorage
format.class=io.confluent.connect.gcs.format.avro.AvroFormat
partitioner.class=io.confluent.connect.storage.partitioner
                  .DefaultPartitioner

schema.compatibility=NONE

confluent.topic.bootstrap.servers=localhost:9092
confluent.topic.replication.factor=1

confluent.license=

该行:

partitioner.class=io.confluent.connect.storage.partitioner
                  .DefaultPartitioner

实际上是这样的:

partitioner.class=io.confluent.connect.storage.partitioner.DefaultPartitioner

由于编辑器的限制,我不得不这样写。抱歉。

根据documentations,在quickstart-gcs.properties中的最后一行实际上是:

confluent.license="" 

但这对我不起作用。

非常感谢您的帮助。

编辑1:按照@ cricket_007注释:

curl http://localhost:8083/connector-plugins

给予:

  

[{“ class”:“ io.confluent.connect.elasticsearch.ElasticsearchSinkConnector”,“ type”:“ sink”,“ version”:“ 5.0.0”},{“ class”:“ io.confluent。 connect.hdfs.HdfsSinkConnector“,” type“:” sink“,” version“:” 5.0.0“},{” class“:” io.confluent.connect.hdfs.tools.SchemaSourceConnector“,” type“:”源”,“版本”:“ 2.0.0-cp1”},{“类”:“ io.confluent.connect.jdbc.JdbcSinkConnector”,“类型”:“接收器”,“版本”:“ 5.0.0” },{“ class”:“ io.confluent.connect.jdbc.JdbcSourceConnector”,“ type”:“ source”,“ version”:“ 5.0.0”},{“ class”:“ io.confluent.connect。 s3.S3SinkConnector“,” type“:”接收器“,”版本“:” 5.0.0“},{” class“:” io.confluent.connect.storage.tools.SchemaSourceConnector“,” type“:”源“ ,“ version”:“ 2.0.0-cp1”},{“ class”:“ org.apache.kafka.connect.file.FileStreamSinkConnector”,“ type”:“ sink”,“ version”:“ 2.0.0- cp1“},{” class“:” org.apache.kafka.connect.file.FileStreamSourceConnector“,” type“:” source“,” version“:” 2.0.0-cp1“}]

0 个答案:

没有答案