Question

我需要在Storm中使用node-client编写弹性搜索螺栓。

我不想使用Transport-client，因为效率较低（节点之间有一个额外的跳跃）。

所以我尝试使用以下代码构建自己的ES bolt：

public void prepare(Map stormConf, TopologyContext context, OutputCollector collector) {

  Settings settings = Settings.settingsBuilder()
    .put("http.enabled", false)
    .put("discovery.zen.ping.multicast.enabled", false)
    .put("discovery.zen.ping.unicast.hosts", hosts)
    .put("discovery.zen.minimum_master_nodes", 1)
    .put("path.home", pathHome)
    .build();

  client = NodeBuilder.nodeBuilder()
    .settings(settings)
    .clusterName(esCluster)
    .data(false)
    .client(true)
    .node().client();
}

我的elastic-search-bolt中的上述方法给出了以下例外：

java.lang.IllegalStateException: failed to load bundle [file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/antlr4-runtime-4.5.1-1.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/asm-5.0.4.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/asm-commons-5.0.4.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/lang-expression-2.2.0.jar, file:/etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/lucene-expressions-5.4.1.jar] due to jar hell

    at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:421) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:115) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.Node.<init>(Node.java:146) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.Node.<init>(Node.java:128) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:152) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at mypkg.storm.NodeClientBolt.prepare(NodeClientBolt.java:89) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at backtype.storm.daemon.executor$fn__5694$fn__5707.invoke(executor.clj:757) ~[storm-core-0.10.0.jar:0.10.0]
    at backtype.storm.util$async_loop$fn__545.invoke(util.clj:477) [storm-core-0.10.0.jar:0.10.0]
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_71]

Caused by: java.lang.IllegalStateException: jar hell!

class: com.esotericsoftware.minlog.Log$Logger
jar1: /etc/storm/storm-0.10.0.nimbus/lib/kryo-2.21.jar
jar2: /etc/storm/storm-0.10.0.nimbus/lib/minlog-1.2.jar

    at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:280) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:186) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:419) ~[storm_es_bolt-0.0.1-SNAPSHOT.jar:?]
    ... 10 more

编辑： Jeremie的好建议解决了上述问题，但后来遇到另一个问题（非常相似）：

Caused by: java.lang.IllegalStateException: jar hell!
class: org.objectweb.asm.AnnotationVisitor
jar1: /etc/storm/storm-0.10.0.nimbus/lib/asm-4.0.jar
jar2: /etc/elasticsearch/elasticsearch-2.2.0/modules/lang-expression/asm-5.0.4.jar
at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:280) ~[storm_hello_world-0.0.1-SNAPSHOT.jar:?]
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:186) ~[storm_hello_world-0.0.1-SNAPSHOT.jar:?]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:419) ~[storm_hello_world-0.0.1-SNAPSHOT.jar:?]
... 10 more

关于如何解决这个问题的任何想法？

提前感谢您的帮助！

Answer 1

Kryo 2.21嵌入（通过树荫插件）库minlog。您可以安全地从您的发行版中删除此jar（/etc/storm/storm-0.10.0.nimbus/lib/minlog-1.2.jar）

在Kryo 3.x中删除了他的依赖项的嵌入

ElasticSearch JarHell com.esotericsoftware.minlog.Log $ Logger

1 个答案: