Question

我想从spark 2.0.0预览版导入库，但是现在maven存储库中没有任何火花2。我在here中使用了建议的解决方案。但它不起作用：错误是：

    The POM for org.apache.spark:spark-core_2.2.11:jar:2.0.0-SNAPSHOT is missing, no dependency information available
The POM for org.apache.spark:spark-mllib_2.11:jar:2.0.0-SNAPSHOT is missing, no dependency information available
The POM for org.apache.spark:spark-sql_2.2.11:jar:2.0.0-SNAPSHOT is missing, no dependency information available
------------------------------------------------------------------------
BUILD FAILURE

The following artifacts could not be resolved: org.apache.spark:spark-core_2.2.11:jar:2.0.0-SNAPSHOT, org.apache.spark:spark-mllib_2.11:jar:2.0.0-SNAPSHOT, org.apache.spark:spark-sql_2.2.11:jar:2.0.0-SNAPSHOT: Could not find artifact org.apache.spark:spark-core_2.2.11:jar:2.0.0-SNAPSHOT -> [Help 1]

我的pom文件看起来像这样：

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.2.11</artifactId>
            <version>2.0.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-mllib_2.11</artifactId>
            <version>2.0.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.2.11</artifactId>
            <version>2.0.0-SNAPSHOT</version>
        </dependency>

Answer 1

将其添加到您的pom.xml：

<repositories>
    <repository>
        <id>asf-snapshot</id>
        <url>http://repository.apache.org/snapshots</url>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

Answer 2

<{> 2.0.0-preview位于Apache Spark Core 2.11。您可以按如下方式修改pom.xml：

<properties>
    <spark.version>2.0.0-preview</spark.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-yarn_2.11</artifactId>
        <version>${spark.version}</version>
    </dependency>

如何在java中使用spark 2.0.0预览版

2 个答案: