要创建的架构的结构:
|-- col1: boolean (nullable = true)
|-- col2: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- col2_1: boolean (nullable = true)
| | |-- col2_2: string (nullable = true)
创建架构的代码:
val prodSchema = StructType(Array(StructField("col1", StringType), StructField("col2",ArrayType(Array(StructField("element",StructType(Array(StructField("col2_1",StringType)))))))))
错误:
found : Array[org.apache.spark.sql.types.StructField]
required: org.apache.spark.sql.types.DataType
StructField("col2",ArrayType(Array(StructField("element",StructType(Array(StructField("col2_1",StringType)))))))
有关如何纠正此架构错误的任何建议。
答案 0 :(得分:0)
您可以使用Schema DSL创建架构:
val col2 = new StructType().add($"col2_1".boolean).add($"col2_2".string)
val schema = new StructType()
.add($"col1".boolean)
.add($"col2".array(col2))
schema.printTreeString()
root
|-- col1: boolean (nullable = true)
|-- col2: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- col2_1: boolean (nullable = true)
| | |-- col2_2: string (nullable = true)
希望有帮助。
答案 1 :(得分:0)
尝试一下:
val schema = StructType(Seq(
StructField("col1",BooleanType,false),
StructField("col2",ArrayType(StructType(Seq(
StructField("col2_1",BooleanType,true),
StructField("col2_2",StringType,true)
)))
)))
答案 2 :(得分:0)
我认为您可以这样写:
val prodSchema =
StructType(
List(
StructField("col1", BooleanType),
StructField("col2", ArrayType(
StructType(
List(
StructField("col2_1", BooleanType),
StructField("col2_2",StringType)
)
)
))
)
)
prodSchema.printTreeString()
root
|-- col1: boolean (nullable = true)
|-- col2: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- col2_1: boolean (nullable = true)
| | |-- col2_2: string (nullable = true)