Question

我正在尝试使用storm-core 1.0.2和kafka-spout 1.0.2运行风暴LocalCluster。

我的pom文件如下所示

  <dependencies>
  <!-- Storm Dependency -->
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-core</artifactId>
        <version>1.0.2</version>
        <scope>provided</scope>
    </dependency>

    <!-- Storm Kafka Dependency -->
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-kafka</artifactId>
        <version>1.0.2</version>
        <exclusions>
            <exclusion>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
 .....
 </dependencies>
<build>
    <resources>
        <resource>
            <directory>src/main/resources</directory>
        </resource>
    </resources>
    <plugins>

        <plugin>
            <artifactId>maven-assembly-plugin</artifactId>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>
                <archive>
                    <manifest>
                        <mainClass>com.walmartlabs.midas.storm.Topology</mainClass>
                    </manifest>
                </archive>
            </configuration>
        </plugin>

        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.2.1</version>
            <executions>
                <execution>
                    <goals>
                        <goal>exec</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <executable>java</executable>
                <includeProjectDependencies>true</includeProjectDependencies>
                <includePluginDependencies>false</includePluginDependencies>
                <classpathScope>compile</classpathScope>
                <mainClass>${storm.topology}</mainClass>
            </configuration>
        </plugin>
    </plugins>
</build>

我使用以下命令在本地运行storm cluster。

mvn package
storm  jar target/myproject-0.1-SNAPSHOT-jar-with-dependencies.jar com.company.project.storm.Topology

我得到以下异常

Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/base/Strings
    at org.apache.storm.kafka.KafkaSpout.declareOutputFields(KafkaSpout.java:184)
    at org.apache.storm.topology.TopologyBuilder.getComponentCommon(TopologyBuilder.java:431)
    at org.apache.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:135)
    at com.walmartlabs.midas.storm.Topology.submitLocalTopology(Topology.java:50)
    at com.walmartlabs.midas.storm.Topology.main(Topology.java:75)
Caused by: java.lang.ClassNotFoundException: com.google.common.base.Strings
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

谷歌的常见依赖是storm-kafka神器。我已经将storm-kafka的范围设置为默认值，所以我相信它的所有依赖都应该在我的jar中构建。

如果我用一个只是读取某个文件的简单喷口替换KafkaSpout，一切正常。我相信在打包jar时我并没有包装storm-kafka artifact的所有依赖。这里有什么问题？

Answer 1

https://github.com/apache/storm/tree/v1.0.2/external/storm-kafka#using-storm-kafka-with-different-versions-of-kafka

Storm 1.0.2将工件“kafka_ [scala_version]”的范围更改为“提供”，这意味着您需要将其添加为依赖项。这是为了在scala版本和/或kafka版本上具有组合的灵活性。

Answer 2

我遇到了这个问题，并通过添加下面的依赖项来解决（ps：guava.jar storm1.0.3使用的版本是16.0.1。顺便说一句，我们可以参考storm-starter使用的番石榴版本。

        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>${guava.version}</version>
            <!--<version>16.0.1</version>-->
        </dependency>

Storm-Kafka：java.lang.NoClassDefFoundError：com / google / common / base / Strings

2 个答案: