elasticsearch5.x flink连接器中的NoClassDefFoundError

时间:2017-08-16 10:47:22

标签: maven pom.xml apache-flink flink-streaming

我遇到了与elasticsearch flink连接器有关的一些问题。 在运行我的程序时,我有很多这样的错误,我很难理解问题出在哪里:

java.lang.NoClassDefFoundError: org/apache/lucene/store/StoreRateLimiting$Type
    at org.elasticsearch.index.store.IndexStoreConfig.<clinit>(IndexStoreConfig.java:40)
    at org.elasticsearch.common.settings.ClusterSettings.<clinit>(ClusterSettings.java:169)
    at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:66)
    at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:138)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:265)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:130)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:116)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:106)
    at org.apache.flink.streaming.connectors.elasticsearch5.Elasticsearch5ApiCallBridge.createClient(Elasticsearch5ApiCallBridge.java:71)
    at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.open(ElasticsearchSinkBase.java:272)
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:111)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:376)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.store.StoreRateLimiting$Type
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 16 more

在pom文件中我添加了:

pom的最后一部分:

<build>
        <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>2.8</version>
            <executions>
                <execution>
                    <id>unpack</id>
                    <!-- executed just before the package phase -->
                    <phase>prepare-package</phase>
                    <goals>
                        <goal>unpack</goal>
                    </goals>
                    <configuration>
                        <artifactItems>
                            <!-- For jarchivelib connector -->
                            <artifactItem>
                                <groupId>org.rauschig</groupId>
                                <artifactId>jarchivelib</artifactId>
                                <version>${jarchivelib.version}</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/rauschig/**</includes>
                            </artifactItem>
                            <!-- For guava library -->
                            <artifactItem>
                                <groupId>com.google.guava</groupId>
                                <artifactId>guava</artifactId>
                                <version>${guava.version}</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>com/google/**</includes>
                            </artifactItem>
                            <!-- For cassandra connector -->
                            <artifactItem>
                                <groupId>org.apache.flink</groupId>
                                <artifactId>flink-connector-cassandra_2.10</artifactId>
                                <version>${flink.version}</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/flink/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>com.codahale.metrics</groupId>
                                <artifactId>metrics-core</artifactId>
                                <version>3.0.2</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>com/codahale/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>com.datastax.cassandra</groupId>
                                <artifactId>cassandra-driver-core</artifactId>
                                <version>${cassandra.version}</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>com/datastax/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.apache.flink</groupId>
                                <artifactId>flink-connector-elasticsearch-base_2.10</artifactId>
                                <version>1.3.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/flink/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.apache.flink</groupId>
                                <artifactId>flink-connector-elasticsearch5_2.10</artifactId>
                                <version>1.3.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/flink/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch.client</groupId>
                                <artifactId>transport</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch</groupId>
                                <artifactId>elasticsearch</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.apache.lucene</groupId>
                                <artifactId>lucene-core</artifactId>
                                <version>6.6.0</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>joda-time</groupId>
                                <artifactId>joda-time</artifactId>
                                <version>2.9.9</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/joda/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch.plugin</groupId>
                                <artifactId>transport-netty3-client</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch.plugin</groupId>
                                <artifactId>transport-netty4-client</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch.plugin</groupId>
                                <artifactId>reindex-client</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch.plugin</groupId>
                                <artifactId>percolator-client</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch.plugin</groupId>
                                <artifactId>lang-mustache-client</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.elasticsearch.plugin</groupId>
                                <artifactId>parent-join-client</artifactId>
                                <version>5.5.1</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/elasticsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>com.carrotsearch</groupId>
                                <artifactId>hppc</artifactId>
                                <version>0.7.2</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>com/carrotsearch/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.apache.logging.log4j</groupId>
                                <artifactId>log4j-core</artifactId>
                                <version>2.8.2</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/logging/log4j/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.apache.logging.log4j</groupId>
                                <artifactId>log4j-api</artifactId>
                                <version>2.8.2</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/logging/log4j/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.apache.httpcomponents</groupId>
                                <artifactId>httpclient</artifactId>
                                <version>4.5.3</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/**</includes>
                            </artifactItem>
                            <artifactItem>
                                <groupId>org.apache.httpcomponents</groupId>
                                <artifactId>httpcore</artifactId>
                                <version>4.4.6</version>
                                <type>jar</type>
                                <overWrite>false</overWrite>
                                <outputDirectory>./target/classes</outputDirectory>
                                <includes>org/apache/**</includes>
                            </artifactItem>
                        </artifactItems>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        </plugins>
    </build>
如果有人可以提供帮助,我将非常高兴。我使用的是flink 1.3.0和elasticsearch5.x!

0 个答案:

没有答案