Question

我正在尝试从Confluent运行kafka-connect-elasticsearch插件，以便将Kafka（ V0.11.0.1 ）中的主题直接流式传输到Elasticsearch中（不需要将Logstash放在其间）。 / p>

我使用Maven构建连接器 -

$ cd kafka-connect-elasticsearch
$ mvn clean package

然后我创建了require配置文件 -

name=es-cluster-lab
connector.class=io.confluent.connect.elasticsearch.ElasticsearchSinkConnector
tasks.max=1
topics=filebeats-test
topic.index.map=filebeats-test:kafka_test_index
key.ignore=true
schema-ignore=true
connection.url=http://elastic:9200
type.name=log

根据新的Kafka Classpath隔离规范，我还在connect-standalone.properties文件中添加了以下行 -

plugin.path=/home/kafka/kafka-connect-elasticsearch-3.3.0/target/kafka-connect-elasticsearch-3.3.0-development/share/java/kafka-connect-elasticsearch/

我去运行脚本......

bin/connect-standalone.sh config/connect-standalone.properties config/elasticsearch-connect.properties

...并收到以下错误。

[2017-09-14 16:08:26,599] INFO Loading plugin from: /home/kafka/kafka-connect-elasticsearch-3.3.0/target/kafka-connect-elasticsearch-3.3.0-development/share/java/kafka-connect-elasticsearch/slf4j-api-1.7.25.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:176)
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;
        at org.reflections.Reflections.expandSuperTypes(Reflections.java:380)
        at org.reflections.Reflections.<init>(Reflections.java:126)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:221)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:198)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:190)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:150)
        at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
        at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:68)

我还尝试将JAR文件移动到/app/kafka/libs directory（默认CLASSPATH），甚至尝试创建子目录/app/kafka/libs/connect_libs并将其手动添加到我的CLASSPATH环境变量中。

除了在Kafka和Elastic之间添加Logstash之外，我还不确定我的下一步是什么。

Answer 1

尝试在构建之前将番石榴版本更改为20

Answer 2

我认为你错过了插件路径末尾的明星＆＃39; *＆＃39; 。

plugin.path=/home/kafka/kafka-connect-elasticsearch-3.3.0/target/kafka-connect-elasticsearch-3.3.0-development/share/java/kafka-connect-elasticsearch/*

Kafka Connect Elasticsearch - NoSuchMethodError

2 个答案: