我正在尝试设置风暴群集,但在群集上运行拓扑时遇到问题。我可以在每台机器上本地运行它们但不能一起运行。我正在使用风暴入门套件中的以下命令来尝试使其工作:
$ storm jar storm-starter-*.jar storm.starter.RollingTopWords production-topology remote
当我在一台主管机器和nimbus机器上运行拓扑时,我遇到了两个不同但相似的错误。我有一个zookeeper实例,我的主管节点应该可以连接到但很明显没有发生。老实说,我甚至不太了解zookeeper如何管理集群。
主管中抛出错误:
scix3@M5:~/apache/storm/examples/storm-starter/target$ storm jar storm-starter-0.9.4.jar storm.starter.RollingTopWords production-topology remote
Running: /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -client - Dstorm.options= -Dstorm.home=/home/scix3/apache/storm -Dstorm.log.dir=/home/scix3/apache/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/scix3/apache/storm/lib/kryo- 2.21.jar:/home/scix3/apache/storm/lib/core.incubator-0.1.0.jar:/home/scix3/apache/storm/lib/commons-fileupload-1.2.1.jar:/home/scix3/apache/storm/lib/ring-servlet- 0.3.11.jar:/home/scix3/apache/storm/lib/clj-stacktrace-0.2.2.jar:/home/scix3/apache/storm/lib/jline-2.11.jar:/home/scix3/apache/storm/lib/servlet-api-2.5.jar:/home/scix3/apache/storm/lib/disruptor-2.10.1.jar:/home/scix3/apache/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/scix3/apache/storm/lib/clojure- 1.5.1.jar:/home/scix3/apache/storm/lib/commons-exec-1.1.jar:/home/scix3/apache/storm/lib/logback-core-1.0.13.jar:/home/scix3/apache/storm/lib/jetty-util-6.1.26.jar:/home/scix3/apache/storm/lib/slf4j-api-1.7.5.jar:/home/scix3/apache/storm/lib/carbonite-1.4.0.jar:/home/scix3/apache/storm/lib/compojure- 1.1.3.jar:/home/scix3/apache/storm/lib/minlog-1.2.jar:/home/scix3/apache/storm/lib/commons-lang-2.5.jar:/home/scix3/apache/storm/lib/tools.macro- 0.1.0.jar:/home/scix3/apache/storm/lib/reflectasm-1.07-shaded.jar:/home/scix3/apache/storm/lib/tools.cli-0.2.4.jar:/home/scix3/apache/storm/lib/math.numeric-tower-0.0.1.jar:/home/scix3/apache/storm/lib/logback-classic-1.0.13.jar:/home/scix3/apache/storm/lib/tools.logging-0.2.3.jar:/home/scix3/apache/storm/lib/asm-4.0.jar:/home/scix3/apache/storm/lib/jetty-6.1.26.jar:/home/scix3/apache/storm/lib/snakeyaml-1.11.jar:/home/scix3/apache/storm/lib/hiccup-0.3.6.jar:/home/scix3/apache/storm/lib/clj-time-0.4.1.jar:/home/scix3/apache/storm/lib/jgrapht-core-0.9.0.jar:/home/scix3/apache/storm/lib/clout-1.0.1.jar:/home/scix3/apache/storm/lib/chill-java-0.3.5.jar:/home/scix3/apache/storm/lib/commons-io- 2.4.jar:/home/scix3/apache/storm/lib/joda-time-2.0.jar:/home/scix3/apache/storm/lib/storm-core-0.9.4.jar:/home/scix3/apache/storm/lib/objenesis-1.2.jar:/home/scix3/apache/storm/lib/commons-logging-1.1.3.jar:/home/scix3/apache/storm/lib/ring-core-1.1.5.jar:/home/scix3/apache/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/scix3/apache/storm/lib/commons-codec-1.6.jar:/home/scix3/apache/storm/lib/json-simple-1.1.jar:/home/scix3/apache/storm/lib/ring-devel-0.3.11.jar:storm-starter-0.9.4.jar:/home/scix3/apache/storm/conf:/home/scix3/apache/storm/bin - Dstorm.jar=storm-starter-0.9.4.jar storm.starter.RollingTopWords production- topology remote
200 [main] INFO storm.starter.RollingTopWords - Topology name: production- topology
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Lists
at storm.starter.tools.Rankings.<init>(Rankings.java:33)
at storm.starter.bolt.AbstractRankerBolt.<init> (AbstractRankerBolt.java:69)
at storm.starter.bolt.AbstractRankerBolt.<init> (AbstractRankerBolt.java:56)
at storm.starter.bolt.IntermediateRankingsBolt.<init> (IntermediateRankingsBolt.java:41)
at storm.starter.RollingTopWords.wireTopology(RollingTopWords.java:68)
at storm.starter.RollingTopWords.<init>(RollingTopWords.java:52)
at storm.starter.RollingTopWords.main(RollingTopWords.java:122)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Lists
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 7 more
scix3@M5:~/apache/storm/examples/storm-starter/target$
从nimbus抛出的错误:
scix3@M6:~/apache/storm/examples/storm-starter/targetstorm jar storm-starter-0.9.4.jar storm.starter.RollingTopWords production-topology remote
Running: java -client -Dstorm.options= -Dstorm.home=/home/scix3/apache/storm -Dstorm.log.dir=/home/scix3/apache/storm/logs - Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/scix3/apache/storm/storm-starter-0.9.4-jar-with- dependencies.jar:/home/scix3/apache/storm/storm-starter-0.9.4.jar:/home/scix3/apache/storm/lib/clj-stacktrace- 0.2.2.jar:/home/scix3/apache/storm/lib/clojure-1.5.1.jar:/home/scix3/apache/storm/lib/minlog- 1.2.jar:/home/scix3/apache/storm/lib/tools.logging-0.2.3.jar:/home/scix3/apache/storm/lib/commons-codec-1.6.jar:/home/scix3/apache/storm/lib/commons-io-2.4.jar:/home/scix3/apache/storm/lib/json-simple-1.1.jar:/home/scix3/apache/storm/lib/asm-4.0.jar:/home/scix3/apache/storm/lib/commons-fileupload-1.2.1.jar:/home/scix3/apache/storm/lib/commons-lang-2.5.jar:/home/scix3/apache/storm/lib/ring-servlet-0.3.11.jar:/home/scix3/apache/storm/lib/carbonite-1.4.0.jar:/home/scix3/apache/storm/lib/jetty-util-6.1.26.jar:/home/scix3/apache/storm/lib/tools.macro-0.1.0.jar:/home/scix3/apache/storm/lib/jgrapht-core-0.9.0.jar:/home/scix3/apache/storm/lib/kryo-2.21.jar:/home/scix3/apache/storm/lib/logback-core-1.0.13.jar:/home/scix3/apache/storm/lib/ring-devel-0.3.11.jar:/home/scix3/apache/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/scix3/apache/storm/lib/logback-classic-1.0.13.jar:/home/scix3/apache/storm/lib/commons-logging-1.1.3.jar:/home/scix3/apache/storm/lib/tools.cli-0.2.4.jar:/home/scix3/apache/storm/lib/reflectasm-1.07-shaded.jar:/home/scix3/apache/storm/lib/math.numeric-tower-0.0.1.jar:/home/scix3/apache/storm/lib/chill-java-0.3.5.jar:/home/scix3/apache/storm/lib/clout-1.0.1.jar:/home/scix3/apache/storm/lib/slf4j-api-1.7.5.jar:/home/scix3/apache/storm/lib/disruptor-2.10.1.jar:/home/scix3/apache/storm/lib/joda-time-2.0.jar:/home/scix3/apache/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/scix3/apache/storm/lib/compojure-1.1.3.jar:/home/scix3/apache/storm/lib/hiccup-0.3.6.jar:/home/scix3/apache/storm/lib/servlet-api-2.5.jar:/home/scix3/apache/storm/lib/commons-exec-1.1.jar:/home/scix3/apache/storm/lib/core.incubator-0.1.0.jar:/home/scix3/apache/storm/lib/jetty-6.1.26.jar:/home/scix3/apache/storm/lib/storm-core-0.9.4.jar:/home/scix3/apache/storm/lib/ring-core-1.1.5.jar:/home/scix3/apache/storm/lib/objenesis-1.2.jar:/home/scix3/apache/storm/lib/jline-2.11.jar:/home/scix3/apache/storm/lib/snakeyaml-1.11.jar:/home/scix3/apache/storm/lib/clj-time-0.4.1.jar:storm-starter-0.9.4.jar:/home/scix3/.storm:/home/scix3/apache/storm/bin -Dstorm.jar=storm-starter-0.9.4.jar storm.starter.RollingTopWords production-topology remote
202 [main] INFO storm.starter.RollingTopWords - Topology name: production-topology
237 [main] INFO storm.starter.RollingTopWords - Running in remote (cluster) mode
Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:38)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:116)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:70)
at storm.starter.util.StormRunner.runTopologyRemotely(StormRunner.java:45)
at storm.starter.RollingTopWords.runRemotely(RollingTopWords.java:78)
at storm.starter.RollingTopWords.main(RollingTopWords.java:129)
Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:83)
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:63)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:47)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:43)
at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
... 5 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 11 more
scix3@M6:~/apache/storm/examples/storm-starter/target$
虽然我已经到目前为止设置风暴和动物园管理员,但我并不真正了解它们的工作方式以及我遇到问题的原因。关于这个问题的任何提示都非常受欢迎!