Question

我想知道DataFrame中的列是否为StructType。我有DataFrame的架构。我正在尝试使用以下代码

df.schema.apply(1) match {
  case StringType => // Do Something
  case ? => // How to check if 1st column is of StructType
}

例如，考虑这种情况：

val personStructType =
  StructType(
    StructField("name", StringType, nullable = true, metadata = new MetadataBuilder().putBoolean("isPrimary", false).build) ::
    StructField("age", IntegerType, nullable = true, metadata = new MetadataBuilder().putBoolean("isPrimary", false).build) ::
    StructField("gender", StringType, nullable = true, metadata = new MetadataBuilder().putBoolean("isPrimary", false).build) ::
    Nil
  )

val idStructType =
  StructType(
    StructField("domain", StringType, nullable = true, metadata = new MetadataBuilder().putBoolean("isPrimary", false).build) ::
    StructField("id", StringType, nullable = true, metadata = new MetadataBuilder().putBoolean("isPrimary", false).build) ::
    Nil
  )

val schema =
  StructType(
    StructField("a", StringType, nullable = true, new MetadataBuilder().putBoolean("isPrimary", true).build) ::
    StructField("person", personStructType, nullable = true, metadata = new MetadataBuilder().putBoolean("isPrimary", false).build) ::
    StructField("identifier", idStructType, nullable = true, metadata = new MetadataBuilder().putBoolean("isPrimary", false).build) ::
    Nil
  )

val a0 = schema.apply(0).dataType
a0 == StringType // Result is true

val a1 = schema.apply(1).dataType
a1 == StructType // Result is false

因为a1是StructType(StructField(name,StringType,true), StructField(age,IntegerType,true), StructField(gender,StringType,true))

我如何知道a1是否为StructType？

Answer 1

当您编写a1 == StructType或case StructType时，您正在与名为StructType的值进行比较，{em> 是类型的伴随对象 StructType。

您需要匹配该类型：case struct: StructType（或case StructType(fields)），就像您撰写case x: String而不是case String一样。

如何检查dataFrame中的列是否为StructType

1 个答案: