Question

我正在尝试对Spark JavaKafkaWordCount.java提供的示例Java代码进行更改，并尝试使用Maven构建它。我正在使用Spark独立模块。我正在使用以下pom.xml：

<project>
    <groupId>com.mycompany.app</groupId>
    <artifactId>my-app</artifactId>
    <modelVersion>4.0.0</modelVersion>
    <name>Simple Project</name>
    <packaging>jar</packaging>
    <version>1.0</version>
    <dependencies>
        <dependency> <!-- Spark dependency -->
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>1.5.1</version>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.8.2</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka_2.10</artifactId>
            <version>1.5.1</version>
        </dependency>
    </dependencies>
</project>

当我跑步时

mvn package

我收到以下错误：

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project my-app: Compilation failure: Compilation failure:
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[17,34] package org.apache.spark.streaming does not exist
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[18,43] package org.apache.spark.streaming.api.java does not exist
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[19,43] package org.apache.spark.streaming.api.java does not exist
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[20,43] package org.apache.spark.streaming.api.java does not exist
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[21,43] package org.apache.spark.streaming.api.java does not exist
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,5] cannot find symbol
[ERROR] symbol:   class JavaStreamingContext
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,37] cannot find symbol
[ERROR] symbol:   class JavaStreamingContext
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,73] cannot find symbol
[ERROR] symbol:   class Duration
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol
[ERROR] symbol:   class JavaPairReceiverInputDStream
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol
[ERROR] symbol:   class JavaPairReceiverInputDStream
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[49,23] cannot access org.apache.spark.streaming.dstream.ReceiverInputDStream
[ERROR] class file for org.apache.spark.streaming.dstream.ReceiverInputDStream not found
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[51,5] cannot find symbol
[ERROR] symbol:   class JavaDStream
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[58,5] cannot find symbol
[ERROR] symbol:   class JavaDStream
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[65,5] cannot find symbol
[ERROR] symbol:   class JavaPairDStream
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

似乎我必须修改我的PATH变量，但我无法找到合适的jar。

Answer 1

您收到的错误消息是：

包org.apache.spark.streaming.api.java不存在

这意味着您缺少以下依赖项：

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming_2.10</artifactId>
    <version>1.5.1</version>
</dependency>

无法导入spark包

1 个答案: