Question

尝试编写SnappySQLJob时出现编译时错误。我错过了依赖吗？

错误消息是：

无法解析org.apache.spark.sql.catalyst.TableIdentifier类型。它是从所需的.class文件间接引用的

@Override
    public Object runJob(Object sparkContext, Config jobConfig) {
        SnappyContext snappyContext = (SnappyContext)sparkContext;

        String fileResource = "data.csv";

        DataFrame dataFrame = snappyContext.read()
                .format("com.databricks.spark.csv")
                .option("header", "true")
                .option("inferSchema", "true")
                .load(fileResource);

        // Compile-Time error is on this line
        dataFrame.write().insertInto("example_table_col");


        return null;
    }

这是我的pom.xml依赖项：

    <dependency>
        <groupId>io.snappydata</groupId>
        <artifactId>snappy-core_2.10</artifactId>
        <version>0.2.1-PREVIEW</version>
    </dependency>
    <dependency>
        <groupId>io.snappydata</groupId>
        <artifactId>snappy-tools_2.10</artifactId>
        <version>0.2.1-PREVIEW</version>
        <exclusions>
            <exclusion>
                <artifactId>jdk.tools</artifactId>
                <groupId>jdk.tools</groupId>
            </exclusion>
            <exclusion>
                <artifactId>logback-classic</artifactId>
                <groupId>ch.qos.logback</groupId>
            </exclusion>
        </exclusions>
    </dependency>

Answer 1

此旧版本似乎缺少spark-catalyst依赖项。我建议改为升级到0.5版本（snappy-tools现在称为snappy-cluster），snappydata集群也应该升级到0.5

对于0.2.1版本，下面应该解决问题：

<dependency>
  <groupId>io.snappydata</groupId>
  <artifactId>snappy-spark-catalyst_2.10</artifactId>
  <version>1.6.0-BETA</version>
</dependency>
<dependency>
  <groupId>io.snappydata</groupId>
  <artifactId>snappy-spark-sql_2.10</artifactId>
  <version>1.6.0-BETA</version>
</dependency>

SnappySQLJob中无法解析org.apache.spark.sql.catalyst.TableIdentifier错误

1 个答案: