Spark 1.6数据框 - 将struct中的所有字段值转换为大写

时间:2018-01-14 03:23:37

标签: scala apache-spark

拥有以下数据框并使用Spark 1.6

scala> dfShow.printSchema
root
 |-- email: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- EML: string (nullable = true)
 |    |    |-- EMT: string (nullable = true)

这可以从源json中读取。如何将所有值转换为大写?在数据框上显示

scala> dfShow.show 
[{"EML":"a.b@c.COM","EMT":"b"},{"EML":"d.e@x.COM","EMT":"h"}]
[{"EML":"g@g.COM","EMT":"h"},{"EML":"f@f.com","EMT":"x"}]
null
[{"EML":"j@j.org","EMT":"b"},{"EML":"r.r@r.COM","EMT":"t"}]

如何将2个字段EML和EMT中的值更改为大写?希望字段看起来如下

[{"EML":"A.B@C.COM","EMT":"B"},{"EML":"D.E@X.COM","EMT":"H"}]
[{"EML":"G@G.COM","EMT":"H"},{"EML":"F@F.com","EMT":"X"}]
null
[{"EML":"J@J.org","EMT":"B"},{"EML":"R.R@R.COM","EMT":"T"}]

这是一个结构域。我有其他结构字段,如地址(line1,line2,city,state,zip,country),phone等。是否有通用的方法将所有struct字段的值转换为upper?

0 个答案:

没有答案