如何使用Java动态展平Spark数据框中的复杂嵌套json

Question

我的输入json数据帧文件如下所示：

company: array (nullable = true)
|    |-- element: struct (containsNull = true)
|    |    |-- address: struct (nullable = true)
|    |    |    |-- city: string (nullable = true)
|    |    |    |-- county: string (nullable = true)
|    |    |    |-- latitude: string (nullable = true)
|    |    |    |-- line1: string (nullable = true)
|    |    |    |-- line2: string (nullable = true)
|    |    |    |-- longitude: string (nullable = true)
|    |    |    |-- postalCode: long (nullable = true)
|    |    |    |-- state: struct (nullable = true)
|    |    |    |    |-- code: int (nullable = true)
|    |    |    |    |-- name: string (nullable = true)
|    |    |    |-- stateOtherDescription: string (nullable = true)
|    |    |-- addressSourceOther: string (nullable = true)
|    |    |-- addressSourceType: struct (nullable = true)
|    |    |     |-- code: int (nullable = true)
|    |    |     |-- name: string (nullable = true)
|    |    |-- reasons: array (nullable = true)
|    |    |     |-- element: struct (containsNull = true)
|    |    |     |-- improve: string (nullable = true)
|    |    |     |-- far: string (nullable = true)
|    |    |     |-- home: string (nullable = true)

我想使用spark java动态展平它。有人可以帮我吗

Answer 1

看看http://stefanfrings.de/bfUtilities/bfUtilities.zip中的de.stefanfrings / parsing / JsonSlurper类。它读取JSON文档，并根据内容创建平面的HashMap。