带有架构的非常大的 DataFrame:
root
|-- id: string (nullable = true)
|-- ext: array (nullable = true)
| |-- element: integer (containsNull = true)
到目前为止,我尝试explode
数据,然后collect_list
:
select
id,
collect_list(cast(item as string))
from default.dual
lateral view explode(ext) t as item
group by
id
但这种方式过于庞大。
答案 0 :(得分:7)
您只需将ext
列转换为字符串数组
df = source.withColumn("ext", source.ext.cast("array<string>"))
df.printSchema()
df.show()