Question

构建递归函数。

def loop(path: String, dt: DataType, acc:Seq[String]): Seq[String] = {
  dt match {
  case s: ArrayType => 
       s.fields.flatMap(f => loop(path + "." + f.name, f.dataType, acc))
  case s: StructType =>      
    s.fields.flatMap(f => loop(path + "." + f.name, f.dataType, acc))
  case other => 
    acc:+ path
}

我有一个错误说“错误：值字段不是org.apache.spark.sql.types.ArrayType的成员”。那么如何迭代arrayType的每个元素并返回一个扁平的字符串序列？

Answer 1

诀窍是使用.elementType

def loop(path: String, dt: DataType, acc:Seq[String]): Seq[String] = {
  dt match {
  case s: ArrayType =>
       loop(path, s.elementType, acc)
  case s: StructType =>      
    s.fields.flatMap(f => loop(path + "." + f.name, f.dataType, acc))
  case other => 
    acc:+ path
}

Spark：如何将spark arrayType作为表达式进行迭代

1 个答案: