Question

代码：

final Session session = connector.openSession();
final PreparedStatement prepared = session.prepare("INSERT INTO spark_test5.messages JSON?");
JavaStreamingContext ssc = new JavaStreamingContext(sc, new Duration(2000)); //error in this line
Map<String, String> kafkaParams = new HashMap<>();
kafkaParams.put("metadata.broker.list", "localhost:9092");

error : 

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/streaming/api/java/JavaStreamingContext
Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.api.java.JavaStreamingContext
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

pom :

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>SparkPoc</groupId>
  <artifactId>Spark-Poc</artifactId>
  <version>0.0.1-SNAPSHOT</version>
 <dependencies>
  <dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming_2.11</artifactId> 
    <version>2.0.0</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.11</artifactId> 
    <version>2.0.1</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming-kafka-0-8_2.11</artifactId> 
    <version>2.0.0</version>
</dependency>
<dependency>
    <groupId>com.datastax.spark</groupId>
    <artifactId>spark-cassandra-connector_2.11</artifactId> <!-- !modified -->
    <version>2.0.0-M3</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.11</artifactId> <!-- !modified -->
    <version>2.0.1</version>
</dependency> 
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.3</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
    </configuration>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-assembly-plugin</artifactId>
    <version>2.4.1</version>
    <configuration>
            <!-- get all project dependencies -->
            <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
            </descriptorRefs>
            <!-- MainClass in mainfest make a executable jar -->
            <archive>
                    <manifest>
                            <mainClass>com.nwf.Consumer</mainClass>
                    </manifest>
            </archive>
    </configuration>
    <executions>
            <execution>
                    <id>make-assembly</id>
                    <!-- bind to the packaging phase -->
                    <phase>package</phase>
                    <goals>
                            <goal>single</goal>
                    </goals>
            </execution>
    </executions>
</plugin>
</plugins>
</build>
</project>

我不知道引入了Java Streaming Context的版本。我在下面包含了代码，错误和pom.xml。我正在尝试集成apache kafka，apache Spark Streaming和Apache Cassandra进行数据处理。

java.lang.NoClassDefFoundError：org / apache / spark / streaming / api / java / JavaStreamingContext

0 个答案: