展平嵌套的json pandas数据帧

时间:2017-02-05 14:26:41

标签: python json python-3.x pandas python-requests

我试图压扁这个数据框:

   allow-live-betting                                        category-id  \
0               False  [10812641776701, 24735152712200, 2583089352210...   
1               False  [10812638253700, 24735152712200, 2473515276510...   
2               False  [10812638253700, 24735152712200, 2475170820040...   
3               False  [10812641776701, 24735152712200, 2583089352210...   
4               False  [10812638253700, 24735152712200, 2473515276510...   
5               False  [10812638253700, 24735152712200, 2475170820040...   
6               False  [10812641776701, 24735152712200, 2583089352210...   
7               False  [10812638253700, 24735152712200, 2473515276510...   
8               False  [10812638253700, 24735152712200, 2475170820040...   
9               False  [10812641776701, 24735152712200, 2583089352210...   
10              False  [10812638253700, 24735152712200, 2473515276510...   
11              False  [10812638253700, 24735152712200, 2475170820040...   
12              False  [10812641776701, 24735152712200, 2583089352210...   
13              False  [10812638253700, 24735152712200, 2473515276510...   
14              False  [10812638253700, 24735152712200, 2475170820040...   
15              False  [10812641776701, 24735152712200, 2583089352210...   
16              False  [10812638253700, 24735152712200, 2473515276510...   
17              False  [10812638253700, 24735152712200, 2475170820040...   
18              False  [10812641776701, 24735152712200, 2583089352210...   
19              False  [10812638253700, 24735152712200, 2473515276510...   

                 id in-running-flag  \
0   402419380850009           False   
1   402419404950010           False   
2   402419422210009           False   
3   402419383670010           False   
4   402419406840009           False   
5   402419426570010           False   
6   402419389770009           False   
7   402419408450010           False   
8   402419431560009           False   
9   402419393790010           False   
10  402419410800009           False   
11  402419434900010           False   
12  402419395470009           False   
13  402419412910010           False   
14  402419437830009           False   
15  402419398400010           False   
16  402419415320009           False   
17  402419443060010           False   
18  402419402560009           False   
19  402419418830010           False   

                                              markets  \
0   [{'in-running-flag': False, 'allow-live-bettin...   
1   [{'in-running-flag': False, 'allow-live-bettin...   
2   [{'in-running-flag': False, 'allow-live-bettin...   
3   [{'in-running-flag': False, 'allow-live-bettin...   
4   [{'in-running-flag': False, 'allow-live-bettin...   
5   [{'in-running-flag': False, 'allow-live-bettin...   
6   [{'in-running-flag': False, 'allow-live-bettin...   
7   [{'in-running-flag': False, 'allow-live-bettin...   
8   [{'in-running-flag': False, 'allow-live-bettin...   
9   [{'in-running-flag': False, 'allow-live-bettin...   
10  [{'in-running-flag': False, 'allow-live-bettin...   
11  [{'in-running-flag': False, 'allow-live-bettin...   
12  [{'in-running-flag': False, 'allow-live-bettin...   
13  [{'in-running-flag': False, 'allow-live-bettin...   
14  [{'in-running-flag': False, 'allow-live-bettin...   
15  [{'in-running-flag': False, 'allow-live-bettin...   
16  [{'in-running-flag': False, 'allow-live-bettin...   
17  [{'in-running-flag': False, 'allow-live-bettin...   
18  [{'in-running-flag': False, 'allow-live-bettin...   
19  [{'in-running-flag': False, 'allow-live-bettin...   

                                            meta-tags               name  \
0   [{'type': 'COUNTRY', 'id': 10812641776701, 'ur...  13:30 Punchestown   
1   [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...  13:40 Musselburgh   
2   [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...      13:50 Taunton   
3   [{'type': 'COUNTRY', 'id': 10812641776701, 'ur...  14:00 Punchestown   
4   [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...  14:10 Musselburgh   
5   [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...      14:20 Taunton   
6   [{'type': 'COUNTRY', 'id': 10812641776701, 'ur...  14:30 Punchestown   
7   [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...  14:40 Musselburgh   
8   [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...      14:50 Taunton   
9   [{'type': 'COUNTRY', 'id': 10812641776701, 'ur...  15:00 Punchestown   
10  [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...  15:10 Musselburgh   
11  [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...      15:20 Taunton   
12  [{'type': 'COUNTRY', 'id': 10812641776701, 'ur...  15:30 Punchestown   
13  [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...  15:40 Musselburgh   
14  [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...      15:50 Taunton   
15  [{'type': 'COUNTRY', 'id': 10812641776701, 'ur...  16:00 Punchestown   
16  [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...  16:10 Musselburgh   
17  [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...      16:20 Taunton   
18  [{'type': 'COUNTRY', 'id': 10812641776701, 'ur...  16:30 Punchestown   
19  [{'type': 'COUNTRY', 'id': 10812638253700, 'ur...  16:40 Musselburgh   

          sport-id                     start status      volume  
0   24735152712200  2017-02-05T13:30:00.000Z   open  6456.48793  
1   24735152712200  2017-02-05T13:40:00.000Z   open  2179.64829  
2   24735152712200  2017-02-05T13:50:00.000Z   open  3411.42369  
3   24735152712200  2017-02-05T14:00:00.000Z   open  3771.21692  
4   24735152712200  2017-02-05T14:10:00.000Z   open   549.20030  
5   24735152712200  2017-02-05T14:20:00.000Z   open  7192.78589  
6   24735152712200  2017-02-05T14:30:00.000Z   open  4048.77794  
7   24735152712200  2017-02-05T14:40:00.000Z   open   449.73440  
8   24735152712200  2017-02-05T14:50:00.000Z   open  1468.62955  
9   24735152712200  2017-02-05T15:00:00.000Z   open  2138.32350  
10  24735152712200  2017-02-05T15:10:00.000Z   open  1159.09601  
11  24735152712200  2017-02-05T15:20:00.000Z   open  3156.49652  
12  24735152712200  2017-02-05T15:30:00.000Z   open   644.83134  
13  24735152712200  2017-02-05T15:40:00.000Z   open   868.18947  
14  24735152712200  2017-02-05T15:50:00.000Z   open  2026.15368  
15  24735152712200  2017-02-05T16:00:00.000Z   open  3307.70675  
16  24735152712200  2017-02-05T16:10:00.000Z   open  5004.96105  
17  24735152712200  2017-02-05T16:20:00.000Z   open   887.37206  
18  24735152712200  2017-02-05T16:30:00.000Z   open  1999.66932  
19  24735152712200  2017-02-05T16:40:00.000Z   open   528.23900  
200
[Finished in 2.1s]

我特别希望“市场”列中的数据排列在不同的列中。数据嵌套在一个树结构中,如xml:

https://matchbook.com/edge/rest/events/?sport-ids=24735152712200

我似乎无法让json_normalise工作。

此代码生成数据框:

r16 = s.get('https://www.matchbook.com/edge/rest/events/?sport-ids=24735152712200')
data3 = r16.json()
df = pd.DataFrame(data3['events'])
print (df)

从这里的答案看来我应该能够做到:

df = json_normalize(data3['markets'])

但没有快乐:(

理想情况下,我希望获取请求将所有信息放在一个数据帧中。

1 个答案:

答案 0 :(得分:0)

这就是你想要的吗?

In [30]: pd.io.json.json_normalize(r16.json()['events'], 'markets').head(3)
Out[30]:
  allow-live-betting  back-overround         event-id               id in-running-flag  lay-overround        market-type name  \
0              False       112.45988  402419389770009  402419392220009           False            NaN  outright_ded_fact  WIN
1              False       102.65156  402419408450010  402419409720010           False      100.32415  outright_ded_fact  WIN
2              False       101.32788  402419431560009  402419433040009           False       99.05799  outright_ded_fact  WIN


     runners                     start status  \
0  [{'prices': [{'available-amount': 300.32832, 'exchange-type': 'back-lay', 'odds-type': 'DECIMAL', 'side': 'back', 'odds': 3.2, 'decimal-o
dds': 3.2...  2017-02-05T14:30:00.000Z   open
1  [{'prices': [{'available-amount': 159.00075, 'exchange-type': 'back-lay', 'odds-type': 'DECIMAL', 'side': 'back', 'odds': 2.47, 'decimal-
odds': 2....  2017-02-05T14:40:00.000Z   open
2  [{'prices': [{'available-amount': 61.19676, 'exchange-type': 'back-lay', 'odds-type': 'DECIMAL', 'side': 'back', 'odds': 4.95, 'decimal-o
dds': 4.9...  2017-02-05T14:50:00.000Z   open

          type       volume
0  multirunner  27405.81060
1  multirunner   3881.65750
2  multirunner   7512.82945