可运行jar中的ElasticSearch：lucene问题

时间：2016-01-28 16:20:08

标签： java maven elasticsearch jar lucene

我正在尝试使用maven创建一个胖的可执行jar，包括elasticsearch作为依赖项来创建TransportClient到正在运行的elasticsearch节点。从eclipse开始，客户端就可以很好地连接到节点，但是当我从整个项目创建一个jar并使用java -jar bla.jar运行时，连接失败了：

DEBUG - [Armor] adding address [{#transport#-1}{127.0.0.1}{127.0.0.1:9300}]
DEBUG - [Armor] connected to node [{#transport#-1}{127.0.0.1}{127.0.0.1:9300}]
INFO  - [Armor] failed to get node info for {#transport#-1}{127.0.0.1}{127.0.0.1:9300}, disconnecting...
 org.elasticsearch.transport.RemoteTransportException: [Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.liveness.LivenessResponse]]
Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.liveness.LivenessResponse]
  at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:179) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:138) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_66-internal]
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_66-internal]
  at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_66-internal]
...
Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.PostingsFormat with name 'Lucene50' does not exist.  You need to add the corresponding JAR file supporting this SPI to your classpath.  The current     classpath supports the following names: [es090, completion090, XBloomFilter]
  at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:109) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.common.lucene.Lucene.<clinit>(Lucene.java:68) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.Version.fromId(Version.java:508) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.Version.readVersion(Version.java:280) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.cluster.node.DiscoveryNode.readFrom(DiscoveryNode.java:327) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.cluster.node.DiscoveryNode.readNode(DiscoveryNode.java:310) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.action.admin.cluster.node.liveness.LivenessResponse.readFrom(LivenessResponse.java:52) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:177) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:138) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[cmd-0.0.1-SNAPSHOT.jar:?]
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_66-internal]
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_66-internal]
  at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_66-internal]

An SPI class of type org.apache.lucene.codecs.PostingsFormat with name 'Lucene50' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [es090, completion090, XBloomFilter]的根本原因。我读到了这一点，它基本上意味着我的最终jar包含来自各种lucene依赖项的相同命名的清单文件，这些文件被彼此覆盖。我使用带有清单资源转换器的maven-shade-plugin来解决问题，但错误仍然存​​在：

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>2.4.3</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <transformers>
                            <transformer
                                implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>de.test.cmd.Main</mainClass>
                            </transformer>
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

然后我找了没有lucene依赖的elasticsearch jar，我只是想创建一个用于通信的transportclient。没有这样的运气，只有一个罐子，包括lucene。我可以做些什么来将elasticsearch放入一个可运行的jar中？或者我应该将所有依赖项保存在jar之外的lib文件夹中？我不知道如何告诉maven这样做

-

更新：当我从eclipse导出对话框中使用“将所需的库复制到jar之外的子文件夹”时，它可以工作。我让eclipse从中生成一个build.xml，但我仍然想尝试使用maven来构建那个jar，或者包含依赖项，或者作为额外的目录。

4 个答案:

答案 0 :(得分：3)

您应该将以下transformer标记添加到阴影插件。

<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>

答案 1 :(得分：1)

我遇到了同样的问题，我发现在maven中更清洁的工作是将lucene-core依赖明确地包含在项目的pom.xml中的elasticsearch依赖项之上。

<dependency>
    <groupId>org.apache.lucene</groupId>
    <artifactId>lucene-core</artifactId>
    <version>5.4.1</version>
 </dependency>
 <dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>2.2.0</version>
 </dependency>

答案 2 :(得分：0)

在我的情况下，是由不正确的

引起的

META-INF/services/{org.apache.lucene.codecs.Codec, org.apache.lucene.codecs.PostingsFormat, org.apache.lucene.codecs.DocValuesFormat}

被错误的库覆盖。

我想有多个库具有与汇编过程中相同的文件，不正确的库会被覆盖。

只需打开正确的jar文件并将相同的文件复制到资源文件夹中即可。我大部分时间都在工作;）

答案 3 :(得分：0)

Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.PostingsFormat with name 'Lucene50' does not exist.  You need to add the corresponding JAR file supporting this SPI to your classpath.  The current classpath supports the following names: [completion, completion090]
at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
at org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
at org.elasticsearch.common.lucene.Lucene.<clinit>(Lucene.java:100)
... 39 more

    <dependency>
        <groupId>org.apache.lucene</groupId>
        <artifactId>lucene-core</artifactId>
        <version>6.6.0</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>5.5.0</version>
    </dependency>
相关问题
最新问题