Kafka Connect Hdfs接收器连接器-找不到io.confluent.connect.hdfs.string.StringFormat类

时间:2018-12-04 18:19:22

标签: apache-kafka hdfs kafka-consumer-api apache-kafka-connect

嗨,我正在尝试使用hdfs接收器连接器将csv数据从kafka移至hdfs,以下是我使用的属性

Connect.properties

name=hdfs-sink
connector.class=io.confluent.connect.hdfs.HdfsSinkConnector
format.class=io.confluent.connect.hdfs.string.StringFormat
tasks.max=1
topics=topic_name
hadoop.conf.dir=/etc/hadoop/conf
hdfs.url=hdfs://nameservice1/dir
flush.size=3
hdfs.authentication.kerberos=true
connect.hdfs.principal=principal
connect.hdfs.keytab=principal.keytab
hdfs.namenode.principal=principal
partitioner.class=io.confluent.connect.hdfs.partitioner.TimeBasedPartitioner
partition.duration.ms=300000
path.format=path.format='year'=YYYY/'month'=MM/'day'=dd
locale=en
timezone=EST

工人属性

bootstrap.servers=kafkaserver
plugin.path=/opt/confluent/share/java
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
offset.storage.file.filename=/tmp/connect.offsets
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
consumer.group.id=connect_group
consumer.auto.offset.reset=earliest

我使用confluent-5.0.1

但是运行kafka connect时出现以下异常

  

java.util.concurrent.ExecutionException:   org.apache.kafka.connect.runtime.rest.errors.BadRequestException:   连接器配置无效,并且包含以下内容1   错误:值无效io.confluent.connect.hdfs.string.StringFormat   用于配置format.class:类   找不到io.confluent.connect.hdfs.string.StringFormat。**   您还可以在端点上找到上述错误列表   / {connectorType} / config / validate在   org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:79)   在   org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:66)   在   org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:110)   造成原因:   org.apache.kafka.connect.runtime.rest.errors.BadRequestException:   连接器配置无效,并且包含以下内容1   错误:值无效io.confluent.connect.hdfs.string.StringFormat   用于配置format.class:类   找不到io.confluent.connect.hdfs.string.StringFormat。您   也可以在端点找到以上错误列表   / {connectorType} / config / validate在   org.apache.kafka.connect.runtime.AbstractHerder.maybeAddConfigErrors(AbstractHerder.java:423)   在   org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:189)   在   org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:107)

0 个答案:

没有答案