Question

我正在尝试对JSON字典中的列表进行数据框化。

我所拥有的： 我有一个来自Twitter API响应的列表中包含多个对象的JSON文件：

[{"created_at": "Wed Dec 19 22:22:50 +0000 2018", "id": 1075516836656136194, "id_str": "1075516836656136194", "text": "@Yahoo Well your about 16 months late on this bombshell: \nhttps://t.co", "display_text_range": [7, 81], "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>", "truncated": false, "in_reply_to_status_id": 1075512357621850112, "in_reply_to_status_id_str": "1075512357621850112", "in_reply_to_user_id": 19380829, "in_reply_to_user_id_str": "19380829", "in_reply_to_screen_name": "Yahoo", "user": {"id": 1075515640302186497, "id_str": "1075515640302186497", "name": "Dave Wasilla", "screen_name": "DaveWasilla", "location": null, "url": null, "description": null, "translator_type": "none", "protected": false, "verified": false, "followers_count": 0, "friends_count": 0, "listed_count": 0, "favourites_count": 0, "statuses_count": 1, "created_at": "Wed Dec 19 22:18:05 +0000 2018", "utc_offset": null, "time_zone": null, "geo_enabled": false, "lang": "en", "contributors_enabled": false, "is_translator": false, "profile_background_color": "F5F8FA", "profile_background_image_url": "", "profile_background_image_url_https": "", "profile_background_tile": false, "profile_link_color": "1DA1F2", "profile_sidebar_border_color": "C0DEED", "profile_sidebar_fill_color": "DDEEF6", "profile_text_color": "333333", "profile_use_background_image": true, "profile_image_url": "http://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png", "profile_image_url_https": "https://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png", "default_profile": true, "default_profile_image": false, "following": null, "follow_request_sent": null, "notifications": null}, "geo": null, "coordinates": null, "place": null, "contributors": null, "quoted_status_id": 1075238373927895040, "quoted_status_id_str": "1075238373927895040", "quoted_status": {"created_at": "Wed Dec 19 03:56:19 +0000 2018", "id": 1075238373927895040, "id_str": "1075238373927895040", "text": "@mkraju \"Turns out?\" Donald Trump Junior testified his father signed this letter of intent back in September of 201\u2026 https://t.co", "display_text_range": [8, 140], "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>", "truncated": true, "in_reply_to_status_id": 1075236204654850048, "in_reply_to_status_id_str": "1075236204654850048", "in_reply_to_user_id": 39155029, "in_reply_to_user_id_str": "39155029", "in_reply_to_screen_name": "mkraju", "user": {"id": 18005085, "id_str": "18005085", "name": "Mollie", "screen_name": "MZHemingway", "location": "Senior Editor, @FDRLST", "url": "http://thefederalist.com", "description": "I don't share your groupthink and it's unlikely I will be bullied into sharing it. Have a great day.", "translator_type": "none", "protected": false, "verified": true, "followers_count": 215017, "friends_count": 1512, "listed_count": 2724, "favourites_count": 11096, "statuses_count": 88844, "created_at": "Tue Dec 09 22:44:12 +0000 2008", "utc_offset": null, "time_zone": null, "geo_enabled": false, "lang": "en", "contributors_enabled": false, "is_translator": false, "profile_background_color": "000000", "profile_background_image_url": "http://abs.twimg.com/images/themes/theme17/bg.gif", "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme17/bg.gif", "profile_background_tile": false, "profile_link_color": "19CF86", "profile_sidebar_border_color": "000000", "profile_sidebar_fill_color": "000000", "profile_text_color": "000000", "profile_use_background_image": false, "profile_image_url": "http://pbs.twimg.com/profile_images/648645685244510208/HD4ui7lh_normal.jpg", "profile_image_url_https": "https://pbs.twimg.com/profile_images/648645685244510208/HD4ui7lh_normal.jpg", "profile_banner_url": "https://pbs.twimg.com/profile_banners/18005085/1541047835", "default_profile": false, "default_profile_image": false, "following": null, "follow_request_sent": null, "notifications": null}, "geo": null, "coordinates": null, "place": null, "contributors": null, "is_quote_status": false, "extended_tweet": {"full_text": "@mkraju \"Turns out?\" Donald Trump Junior testified his father signed this letter of intent back in September of 2017. Link here (https://t.co) Relevant portion of testimony shown. https://t.co", "display_text_range": [8, 190], "entities": {"hashtags": [], "urls": [{"url": "https://t.co", "expanded_url": "https://www.judiciary.senate.gov/imo/media/doc/Trump%20Jr%20Transcript_redacted.pdf", "display_url": "judiciary.senate.gov/imo/media/doc/\u2026", "indices": [129, 152]}], "user_mentions": [{"screen_name": "mkraju", "name": "Manu Raju", "id": 39155029, "id_str": "39155029", "indices": [0, 7]}], "symbols": [], "media": [{"id": 1075238230591791105, "id_str": "1075238230591791105", "indices": [191, 214], "media_url": "http://pbs.twimg.com/media/DuwDeU9X4AEFRbt.jpg", "media_url_https": "https://pbs.twimg.com/media/DuwDeU9X4AEFRbt.jpg", "url": "https://t.co", "display_url": "pic.twitter.com/VLBpb7YkEr", "expanded_url": "https://twitter.com/MZHemingway/status/1075238373927895040/photo/1", "type": "photo", "sizes": {"small": {"w": 371, "h": 680, "resize": "fit"}, "thumb": {"w": 150, "h": 150, "resize": "crop"}, "large": {"w": 532, "h": 975, "resize": "fit"}, "medium": {"w": 532, "h": 975, "resize": "fit"}}}]}, "extended_entities": {"media": [{"id": 1075238230591791105, "id_str": "1075238230591791105", "indices": [191, 214], "media_url": "http://pbs.twimg.com/media/DuwDeU9X4AEFRbt.jpg", "media_url_https": "https://pbs.twimg.com/media/DuwDeU9X4AEFRbt.jpg", "url": "https://t.co", "display_url": "pic.twitter.com/VLBpb7YkEr", "expanded_url": "https://twitter.com/MZHemingway/status/1075238373927895040/photo/1", "type": "photo", "sizes": {"small": {"w": 371, "h": 680, "resize": "fit"}, "thumb": {"w": 150, "h": 150, "resize": "crop"}, "large": {"w": 532, "h": 975, "resize": "fit"}, "medium": {"w": 532, "h": 975, "resize": "fit"}}}]}}, "quote_count": 62, "reply_count": 73, "retweet_count": 203, "favorite_count": 487, "entities": {"hashtags": [], "urls": [{"url": "https://t.co", "expanded_url": "https://twitter.com/i/web/status/1075238373927895040", "display_url": "twitter.com/i/web/status/1\u2026", "indices": [117, 140]}], "user_mentions": [{"screen_name": "mkraju", "name": "Manu Raju", "id": 39155029, "id_str": "39155029", "indices": [0, 7]}], "symbols": []}, "favorited": false, "retweeted": false, "possibly_sensitive": false, "filter_level": "low", "lang": "en"}, "quoted_status_permalink": {"url": "https://t.co", "expanded": "https://twitter.com/MZHemingway/status/1075238373927895040", "display": "twitter.com/MZHemingway/st\u2026"}, "is_quote_status": true, "quote_count": 0, "reply_count": 0, "retweet_count": 0, "favorite_count": 0, "entities": {"hashtags": [], "urls": [{"url": "https://t.co", "expanded_url": "https://twitter.com/MZHemingway/status/1075238373927895040", "display_url": "twitter.com/MZHemingway/st\u2026", "indices": [58, 81]}], "user_mentions": [{"screen_name": "Yahoo", "name": "Yahoo", "id": 19380829, "id_str": "19380829", "indices": [0, 6]}], "symbols": []}, "favorited": false, "retweeted": false, "possibly_sensitive": false, "filter_level": "low", "lang": "en", "timestamp_ms": "1545258170652"}, {another JSON object},{another object}]

我想要的：

我需要为该文件中存在的每个对象的“实体”字典中的“ URL”列表中的以下数据框：

"entities": {"hashtags": [], "urls": [{"url": "https://t.co", "expanded_url": "https://twitter.com/i/web/status/1075238373927895040", "display_url": "twitter.com/i/web/status/1\u2026", "indices": [117, 140]}], "user_mentions": [{"screen_name": "mkraju", "name": "Manu Raju", "id": 39155029, "id_str": "39155029", "indices": [0, 7]}], "symbols": []}

我做了什么：

我尝试了以下代码：

import pandas as pd
import json
from pandas.io.json import json_normalize
with open('test.json') as data_file:    
    data = json.load(data_file)
df = pd.io.json.json_normalize(data)

我得到的：

它显示了数据框表中的以下列（实体列，列表以字符串形式）：

| entities.urls |
  ________ 
[{'url': 'https://t.co', 'expanded_...

如何正确执行此操作？谢谢！

编辑：

@GregD的回答启发了我：

我尝试了这段代码，我快到了：

import json
import pandas as pd
p = 'test.json'
with open(p, 'r') as f:
    doc = json.loads(f.read())
pd.DataFrame.from_dict(doc[0]['entities']['urls'])

现在我需要为每个索引得到相同的答案。请帮忙！

Answer 1

如果我理解正确，则需要一个表格，其中每一行都是一个实体，而各列是不同的url，如下所示：

想象你有：

entity = {"hashtags": [], "urls": [{"url": "https://t.co", "expanded_url": "https://twitter.com/i/web/status/1075238373927895040", "display_url": "twitter.com/i/web/status/1\u2026", "indices": [117, 140]}], "user_mentions": [{"screen_name": "mkraju", "name": "Manu Raju", "id": 39155029, "id_str": "39155029", "indices": [0, 7]}], "symbols": []}

然后：

import pandas as pd

pd.DataFrame.from_dict(entity['urls'])

应该给你。

JSON文件中的JSON词典中的数据框列表（python-pandas）

1 个答案: