Scala - 存储在数据集中的类型缺少编码器

时间:2018-03-06 03:29:46

标签: scala apache-spark encoder

我正在尝试在Scala 2.2中运行以下命令

   val x_test0 = cn_train.map( { case row => row.toSeq.toArray } )

我一直遇到以下错误

 error: Unable to find encoder for type stored in a Dataset.  Primitive types (Int, String, etc) and Product types (case classes) are supported by importing spark.implicits._  Support for serializing other types will be added in future releases.

我已经通过以下命令导入了implicits._:

val spark = SparkSession.builder.getOrCreate()
import spark.implicits._

1 个答案:

答案 0 :(得分:1)

该错误消息告诉您,它无法找到异类Encoder的{​​{1}}以将其保存在Array中。但是你可以得到Dataset RDD这样的内容:

Array