Question

我将以下Dataframe作为示例：

+--------------------------------------+------------+------------+------------------+
|               user_id                |    city    | user_name  |   facebook_id    |
+--------------------------------------+------------+------------+------------------+
| 55c3c59d-0163-46a2-b495-bc352a8de883 | Toronto    | username_x | 0123482174440907 |
| e2ddv22d-4132-c211-4425-9933aa8de454 | Washington | username_y | 0432982476780234 |
+--------------------------------------+------------+------------+------------------+

如何将其转换为JSON Objecta数组，如：

[{
  "user_id": "55c3c59d-0163-46a2-b495-bc352a8de883",
  "facebook_id": "0123482174440907"
},
{
  "user_id": "e2ddv22d-4132-c211-4425-9933aa8de454",
  "facebook_id": "0432982476780234"
}]

Answer 1

假设您已经以数据帧的形式加载了给定数据，则可以在数据帧上使用函数toJSON。

*scala> sc.parallelize(Seq(("55c3c59d-0163-46a2-b495-bc352a8de883","Toronto","username_x","0123482174440907"))).toDF("user_id","city","user_name","facebook_id")
res2: org.apache.spark.sql.DataFrame = [user_id: string, city: string, user_name: string, facebook_id: string]*


*res2.toJSON.take(1)
res3: Array[String] = Array({"user_id":"55c3c59d-0163-46a2-b495-bc352a8de883","city":"Toronto","user_name":"username_x","facebook_id":"0123482174440907"})*

将Spark数据帧转换为JSON对象数组

1 个答案: