Question

我正在尝试在HDP网站上执行有关Storm Ingestion的教程。

当我执行程序时，但我收到以下错误：

java.lang.NoClassDefFoundError：storm.kafka.DynamicBrokersReader上的org / json / simple / JSONValue

但是，我看到类JSONValue存在于项目jar中的json-simple-1.1 jar中。

我在Linux中执行主Jar（Tutorial-1.0-SNAPSHOT.jar）。

任何想法，NoClassDefFoundError背后可能是什么原因？

这是我的POM文件。

enter code here

<?xml version="1.0"?>
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0     http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

<modelVersion>4.0.0</modelVersion>
<groupId>tutorial</groupId>
<artifactId>Tutorial</artifactId>
<version>1.0-SNAPSHOT</version>
<name>TruckEventProcessing</name>
<url>http://maven.apache.org</url>
<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <storm.version>0.10.0.2.3.0.0-2557</storm.version>
    <storm.kafka.version>0.10.0.2.3.0.0-2557</storm.kafka.version>
    <hadoop.version>2.7.1.2.3.0.0-2557</hadoop.version>
    <hbase.version>1.1.1.2.3.0.0-2557</hbase.version>
    <log4j.version>1.2.17</log4j.version>
    <storm.hdfs.version>0.10.0.2.3.0.0-2557</storm.hdfs.version>
    <storm.hbase.version>0.10.0.2.3.0.0-2557</storm.hbase.version>
    <hive.version>1.2.1.2.3.0.0-2557</hive.version>

</properties>

<repositories>
    <repository>
        <id>hortonworks</id>
        <!-- <url>http://repo.hortonworks.com/content/repositories/releases/</url> -->
        <url>http://nexus-private.hortonworks.com/nexus/content/groups/public/</url>
    </repository>
</repositories>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>1.4</version>
            <configuration>
                <createDependencyReducedPom>true</createDependencyReducedPom>
                <filters>
                    <filter>
                        <artifact>*:*</artifact>
                        <excludes>
                            <exclude>META-INF/*.SF</exclude>
                            <exclude>META-INF/*.DSA</exclude>
                            <exclude>META-INF/*.RSA</exclude>
                        </excludes>
                    </filter>
                </filters>
            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>

                        <transformers>
                            <transformer
                                implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                            <transformer
                                implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass></mainClass>
                            </transformer>
                        </transformers>


                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>


    <resources>
        <resource>
            <directory>/etc/hbase/conf</directory>
            <includes>
                <include>hbase-site.xml</include>
            </includes>
        </resource>

        <resource>
            <directory>/opt/TruckEvents/Tutorials-master/src/main/resources</directory>
        </resource>

    </resources>

</build>
<dependencies>
    <!-- Storm Dependencies -->
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-core</artifactId>
        <version>${storm.version}</version>
        <scope>compile</scope>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>log4j-over-slf4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>

        </exclusions>
    </dependency>

    <!-- Storm Kafka Dependencies -->
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-kafka</artifactId>
        <version>${storm.kafka.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>


    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-hbase</artifactId>
        <version>${storm.hbase.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!-- Storm HDFS Dependencies -->
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-hdfs</artifactId>
        <version>${storm.hdfs.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>


    <!-- Hadoop Dependencies -->

    <dependency>
        <groupId>xerces</groupId>
        <artifactId>xercesImpl</artifactId>
        <version>2.9.1</version>
    </dependency>

    <dependency>
        <groupId>xalan</groupId>
        <artifactId>xalan</artifactId>
        <version>2.7.1</version>
    </dependency>

    <dependency>
        <groupId>org.htrace</groupId>
        <artifactId>htrace-core</artifactId>
        <version>3.0.4</version>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-hdfs</artifactId>
        <version>${hadoop.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.101tec</groupId>
        <artifactId>zkclient</artifactId>
        <version>0.5-1</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>


    <!-- Jetty Dependencies -->
    <dependency>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty</artifactId>
        <version>6.1.26</version>
    </dependency>


    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.10</artifactId>
        <version>0.8.2.2.3.0.0-2557</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>



    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>${hadoop.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!-- HBase Dependcies -->
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-client</artifactId>
        <version>${hbase.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>log4j-over-slf4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!-- Twitter4j Dependcies -->

    <dependency>
        <groupId>org.twitter4j</groupId>
        <artifactId>twitter4j-core</artifactId>
        <version>4.0.2</version>
    </dependency>

    <!-- Hive dependencies -->
    <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-exec</artifactId>
        <version>${hive.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>log4j-over-slf4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!-- Other Dependency -->
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>${log4j.version}</version>
    </dependency>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>3.8.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Answer 1

在0.10.0 Storm重定位类中减少依赖冲突，请参阅 http://storm.apache.org/2015/06/15/storm0100-beta-released.html

此外，json.simple包已重新定位，请参阅https://github.com/apache/storm/blob/v0.10.0-beta/storm-core/pom.xml#L552。所以过去工作的代码，因为它隐含地依赖于这个包的存在，现在将失败。为了解决这个问题，我将以下依赖项添加到我的pom文件中：

<dependency>
  <groupId>com.googlecode.json-simple</groupId>
  <artifactId>json-simple</artifactId>
  <version>1.1.1</version>
</dependency>

在storm.kafka.DynamicBrokersReader上的java.lang.NoClassDefFoundError：org / json / simple / JSONValue

1 个答案: