Question

我已经在使用Spark 1.6.1并且现在正在评估Spark 2.0 Preview，但我找不到org.apache.spark.sql.Row。

这是必需的，因为我正在将我的DataFrame代码迁移到1.6.1到2.0预览中。我在这里错过了一些东西吗？我的maven依赖关系粘贴在

之下

  <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
        <version>2.0.0-preview</version>
         <scope>system</scope>
        <systemPath>C://spark-2.0.0-preview-bin-hadoop2.7//jars//spark-core_2.11-2.0.0-preview.jar</systemPath>
    </dependency>
    <dependency>
        <groupId>com.oracle</groupId>
        <artifactId>ojdbc7</artifactId>
        <version>12.1.0.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_2.11</artifactId>
        <version>2.0.0-preview</version>
         <scope>system</scope>
        <systemPath>C://spark-2.0.0-preview-bin-hadoop2.7//jars//spark-sql_2.11-2.0.0-preview.jar</systemPath>
    </dependency>

Answer 1

在spark v2.0.0中，Row已移至另一个jar文件，将此添加到您的maven依赖

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-catalyst_2.11</artifactId>
    <version>2.0.0-preview</version>
    <scope>system</scope>
    <systemPath>C://spark-2.0.0-preview-bin-hadoop2.7//spark-catalyst_2.11-2.0.0-preview.jar</systemPath>
</dependency>

Answer 2

使用它，它对我有用。从Spark 1.6.1迁移到2.0 对于Maven，

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-tags_2.11</artifactId>
    <version>2.0.0-preview</version>
</dependency>

对于SBT，

libraryDependencies += "org.apache.spark" % "spark-tags_2.11" % "2.0.0-preview"

对于Gradle，

compile group: 'org.apache.spark', name: 'spark-tags_2.11', version: '2.0.0-preview'

无法在Spark 2.0 Preview中解析org.apache.spark.sql.Row

2 个答案: