Question

我正在尝试创建案例类对象的RDD。例如，

// sqlContext from the previous example is used in this example.
// createSchemaRDD is used to implicitly convert an RDD to a SchemaRDD.
import sqlContext.createSchemaRDD

val people: RDD[Person] = ... // An RDD of case class objects, from the previous example.

// The RDD is implicitly converted to a SchemaRDD by createSchemaRDD, allowing it to be stored using        Parquet.
people.saveAsParquetFile("people.parquet")

我试图通过提供

来完成上一个例子中的部分

    case class Person(name: String, age: Int)

    // Create an RDD of Person objects and register it as a table.
    val people: RDD[Person] = sc.textFile("/user/root/people.txt").map(_.split(",")).map(p => Person(p(0), p(1).trim.toInt))
    people.registerAsTable("people")

我收到以下错误：

<console>:28: error: not found: type RDD
       val people: RDD[Person] =sc.textFile("/user/root/people.txt").map(_.split(",")).map(p => Person(p(0), p(1).trim.toInt))

对于出了什么问题有什么想法吗？提前谢谢！

Answer 1

此处的问题是显式RDD[String]类型注释。看起来RDD默认情况下未导入spark-shell，这就是Scala抱怨无法找到RDD类型的原因。首先尝试运行import org.apache.spark.rdd.RDD。

创建RDD时找不到火花错误RDD类型

1 个答案: