Question

我有json数据，我想将json数据转换为dataframe

[
{FlierNumber:,BaggageTypeReturn:,FirstName:K,Title:1,MiddleName:D,LastName:Gupta,MealTypeOnward:,DateOfBirth:,BaggageTypeOnward:,SeatTypeOnward:,MealTypeReturn:,FrequentAirline:null,Type:A,SeatTypeReturn:},
{FlierNumber:,BaggageTypeReturn:,FirstName:Sweety,Title:2,MiddleName:,LastName:Gupta,MealTypeOnward:,DateOfBirth:,BaggageTypeOnward:,SeatTypeOnward:,MealTypeReturn:,FrequentAirline:null,Type:A,SeatTypeReturn:}
]

Answer 1

您在上面提供的JSON无效。这是语法正确的JSON格式

[{"FlierNumber":"","BaggageTypeReturn":"","FirstName":"K","Title":"1","MiddleName":"D","LastName":"Gupta","MealTypeOnward":"","DateOfBirth":"","BaggageTypeOnward":"","SeatTypeOnward":"","MealTypeReturn":"","FrequentAirline":"null","Type":"A","SeatTypeReturn":""},{"FlierNumber":"","BaggageTypeReturn":"","FirstName":"Sweety","Title":"2","MiddleName":"","LastName":"Gupta","MealTypeOnward":"","DateOfBirth":"","BaggageTypeOnward":"","SeatTypeOnward":"","MealTypeReturn":"","FrequentAirline":"null","Type":"A","SeatTypeReturn":""}]

如果文件中存在该文件，则可以使用

直接读取spark

  val jsonDF = spark.read.json("filepath\sample.json")
  jsonDF.printSchema()
  jsonDF.show

结果是：

root
 |-- BaggageTypeOnward: string (nullable = true)
 |-- BaggageTypeReturn: string (nullable = true)
 |-- DateOfBirth: string (nullable = true)
 |-- FirstName: string (nullable = true)
 |-- FlierNumber: string (nullable = true)
 |-- FrequentAirline: string (nullable = true)
 |-- LastName: string (nullable = true)
 |-- MealTypeOnward: string (nullable = true)
 |-- MealTypeReturn: string (nullable = true)
 |-- MiddleName: string (nullable = true)
 |-- SeatTypeOnward: string (nullable = true)
 |-- SeatTypeReturn: string (nullable = true)
 |-- Title: string (nullable = true)
 |-- Type: string (nullable = true)


+-----------------+-----------------+-----------+---------+-----------+---------------+--------+--------------+--------------+----------+--------------+--------------+-----+----+
|BaggageTypeOnward|BaggageTypeReturn|DateOfBirth|FirstName|FlierNumber|FrequentAirline|LastName|MealTypeOnward|MealTypeReturn|MiddleName|SeatTypeOnward|SeatTypeReturn|Title|Type|
+-----------------+-----------------+-----------+---------+-----------+---------------+--------+--------------+--------------+----------+--------------+--------------+-----+----+
|                 |                 |           |        K|           |           null|   Gupta|              |              |         D|              |              |    1|   A|
|                 |                 |           |   Sweety|           |           null|   Gupta|              |              |          |              |              |    2|   A|
+-----------------+-----------------+-----------+---------+-----------+---------------+--------+--------------+--------------+----------+--------------+--------------+-----+----+

如何将嵌套的json隐藏到datafarme

1 个答案: