如何从数据框字典创建数据框?

时间:2018-12-30 03:01:45

标签: python python-3.x pandas jupyter-notebook data-science

问题是我想从一个递归嵌套的JSON文件创建一个Dataframe,然后我想出了自己的方法。如果您还有其他更简单的方法,请告诉我

我在json文件中记录了24小时的飞行数据,并想使用它创建一个数据框来形成一天的数据框。我可以通过每小时创建一个对象然后通过使用append合并所有对象来做到这一点,但是即使是24个小时,这似乎也是一个乏味的过程,如果我有1000条记录要以这种方式合并的话该怎么办。因此,我想出了以下代码- 我尝试遍历每小时数据的JSON文件并一次创建两个数据框,并使用concat合并它们。稍后,我将几个数据帧存储在字典中,并用变量z指示键值。当我打印它时,输出一次包含2条记录,并被打印12次,因此使其在24小时内全部成为数据,但是当我将数据存储在csv文件中时,仅存储与最近两个小时相对应的数据。

    import pandas as pd
    z = 0
    result1 = {}
    for i in range(0,24):
        if (i%2)==0:
            dataF = data["data"]["weather"][0]["hourly"][i]
            dataF2 = data["data"]["weather"][0]["hourly"][i+1]
            dataFrame = pd.DataFrame(dataF,index=[i+1])
            dataFrame2 = pd.DataFrame(dataF2,index=[i+2])
            frame = [dataFrame,dataFrame2]
            result = pd.concat(frame)
            #result.append(result)
            #print(result1)
            result1[z] = result
            z += 1

    print(result1)`

打印结果:

{0:   windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles  \
1            11         46             NW          8         8              7   
2            13         42             NW          6         7              8   

  DewPointF HeatIndexF cloudcover HeatIndexC ...   \
1        47         49        100         10 ...    
2        44         46        100          8 ...    

                         weatherDesc visibility weatherCode tempC tempF  \
1  {'value': 'Patchy rain possible'}         10         176    10    49   
2  {'value': 'Patchy rain possible'}         10         176     8    46   

  WindChillF WindChillC winddirDegree humidity time  
1         46          8           318       91    0  
2         42          6           317       92  100  

[2 rows x 25 columns], 1:   windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles  \
3            14         38             NW          3         5              9   
4            16         33             NW          1         3             10   

  DewPointF HeatIndexF cloudcover HeatIndexC ...             weatherDesc  \
3        41         43        100          6 ...   {'value': 'Overcast'}   
4        38         40        100          4 ...   {'value': 'Overcast'}   

  visibility weatherCode tempC tempF WindChillF WindChillC winddirDegree  \
3         10         122     6    43         38          3           315   
4         10         122     4    40         33          1           314   

  humidity time  
3       92  200  
4       93  300  

[2 rows x 25 columns], 2:   windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles  \
5            17         33             NW          1         3             10   
6            17         33             NW          0         3             11   

  DewPointF HeatIndexF cloudcover HeatIndexC ...             weatherDesc  \
5        38         40        100          4 ...   {'value': 'Overcast'}   
6        38         39        100          4 ...   {'value': 'Overcast'}   

  visibility weatherCode tempC tempF WindChillF WindChillC winddirDegree  \
5         10         122     4    40         33          1           314   
6         10         122     4    40         33          0           315   

  humidity time  
5       93  400  
6       93  500  

[2 rows x 25 columns], 3:   windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC windspeedMiles  \
7            18         32             NW          0         3             11   
8            18         33             NW          1         3             11   

  DewPointF HeatIndexF cloudcover HeatIndexC ...             weatherDesc  \
7        38         39        100          4 ...   {'value': 'Overcast'}   
8        38         40        100          4 ...   {'value': 'Overcast'}   

  visibility weatherCode tempC tempF WindChillF WindChillC winddirDegree  \
7         10         122     4    40         32          0           315   
8         10         122     5    41         33          1           317   

  humidity time  
7       93  600  
8       93  700  

[2 rows x 25 columns], 4:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
9             17         34             NW          1         4   
10            17         35             NW          2         4   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC ...   \
9              11        38         40        100          5 ...    
10             11        39         41        100          5 ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
9   {'value': 'Overcast'}         10         122     5    42         34   
10  {'value': 'Overcast'}         10         122     6    42         35   

   WindChillC winddirDegree humidity time  
9           1           318       92  800  
10          2           319       92  900  

[2 rows x 25 columns], 5:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
11            17         36             NW          2         4   
12            17         37             NW          3         4   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC  ...   \
11             10        38         42        100          6  ...    
12             10        38         43        100          6  ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
11  {'value': 'Overcast'}         10         122     7    44         36   
12  {'value': 'Overcast'}         10         122     7    45         37   

   WindChillC winddirDegree humidity  time  
11          2           318       88  1000  
12          3           317       84  1100  

[2 rows x 25 columns], 6:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
13            17         38             NW          3         3   
14            17         38             NW          4         3   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC  ...   \
13             10        38         44        100          7  ...    
14             10        38         44        100          7  ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
13  {'value': 'Overcast'}         10         122     8    47         38   
14  {'value': 'Overcast'}         10         122     9    47         38   

   WindChillC winddirDegree humidity  time  
13          3           317       81  1200  
14          4           317       79  1300  

[2 rows x 25 columns], 7:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
15            17         39             NW          4         3   
16            17         39             NW          4         3   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC  ...   \
15             10        38         44        100          7  ...    
16             11        37         45        100          7  ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
15  {'value': 'Overcast'}         10         122     9    48         39   
16  {'value': 'Overcast'}         10         122    10    49         39   

   WindChillC winddirDegree humidity  time  
15          4           317       76  1400  
16          4           317       74  1500  

[2 rows x 25 columns], 8:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
17            17         39             NW          4         3   
18            17         39             NW          4         2   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC  ...   \
17             11        37         44        100          7  ...    
18             11        36         44        100          7  ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
17  {'value': 'Overcast'}         10         122     9    48         39   
18  {'value': 'Overcast'}         10         122     8    46         39   

   WindChillC winddirDegree humidity  time  
17          4           318       74  1600  
18          4           318       73  1700  

[2 rows x 25 columns], 9:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
19            17         38             NW          3         2   
20            17         37             NW          3         2   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC  ...   \
19             11        36         44        100          7  ...    
20             11        35         43        100          6  ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
19  {'value': 'Overcast'}         10         122     7    45         38   
20  {'value': 'Overcast'}         10         122     7    44         37   

   WindChillC winddirDegree humidity  time  
19          3           319       73  1800  
20          3           318       72  1900  

[2 rows x 25 columns], 10:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
21            17         37             NW          3         1   
22            16         36             NW          2         1   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC  ...   \
21             10        34         43        100          6  ...    
22             10        33         42        100          6  ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
21  {'value': 'Overcast'}         10         122     6    43         37   
22  {'value': 'Overcast'}         10         122     6    42         36   

   WindChillC winddirDegree humidity  time  
21          3           317       71  2000  
22          2           316       71  2100  

[2 rows x 25 columns], 11:    windspeedKmph FeelsLikeF winddir16Point FeelsLikeC DewPointC  \
23            16         35             NW          2         1   
24            16         35             NW          2         1   

   windspeedMiles DewPointF HeatIndexF cloudcover HeatIndexC  ...   \
23             10        33         41         95          5  ...    
24             10        33         41         91          5  ...    

              weatherDesc visibility weatherCode tempC tempF WindChillF  \
23  {'value': 'Overcast'}         10         122     5    41         35   
24  {'value': 'Overcast'}         10         122     5    41         35   

   WindChillC winddirDegree humidity  time  
23          2           317       72  2200  
24          2           318       74  2300  

[2 rows x 25 columns]}

1 个答案:

答案 0 :(得分:1)

仅以您的示例为例:

如果您将整个json作为字典(或列表)加载,例如使用json.load,可以使用json_normalize:

>>> import pandas as pd
>>> import json
>>> from pandas.io.json import json_normalize

数据:

>>> d = {"response": {"body": {"contact": {"email": "mr@abc.com", "mobile_number": "0123456789"}, "personal": {"last_name": "Muster", "gender": "m", "first_name": "Max", "dob": "1985-12-23", "family_status": "single", "title": "Dr."}, "customer": {"verified": "true", "customer_id": "1234567"}}, "token": "dsfgf", "version": "1.1"}}
>>> df = pd.io.json.json_normalize(d)
>>> df
  response.body.contact.email response.body.contact.mobile_number response.body.customer.customer_id       ...        response.body.personal.title response.token response.version
0                  mr@abc.com                          0123456789                            1234567       ...                                 Dr.          dsfgf              1.1

[1 rows x 12 columns]
>>> df.columns = df.columns.map(lambda x: x.split(".")[-1])
>>> df
        email mobile_number customer_id verified         dob family_status first_name gender last_name title  token version
0  mr@abc.com    0123456789     1234567     true  1985-12-23        single        Max      m    Muster   Dr.  dsfgf     1.1

看看here

另一个post