Question

How do I...

1) Parse JSON objects from API queries in Python 3

2) Parse multiple requests into a list, and

3) Output the list into a JSON file

Answer 1

我更喜欢使用requests进行所有API编程。这是一个单行程序，它获取几个API调用的结果，将它们放在一个列表中，并将该列表写入JSON文件：

json.dump([requests.get(url).json() for url in URLs], fp)

这是一个完整的测试程序：

import requests
import json

URLs = [
    # Some URLs that return JSON objects
    'http://httpbin.org/ip',
    'http://httpbin.org/user-agent',
    'http://httpbin.org/headers'
]

with open('result.json', 'w') as fp:
    json.dump([requests.get(url).json() for url in URLs], fp, indent=2)

如果您因某种原因对requests过敏，这里只使用标准库，这是等效的Python3代码。

from urllib.request import urlopen
import json

URLs = [
    # Some URLs that return JSON objects
    'http://httpbin.org/ip',
    'http://httpbin.org/user-agent',
    'http://httpbin.org/headers'
]

json_list = []
for url in URLs:
    resp = urlopen(url)
    resp = resp.read().decode(resp.headers.get_content_charset() or 'ascii')
    json_list.append(json.loads(resp))
with open('result.json', 'w') as fp:
    json.dump(json_list, fp, indent=2)

Answer 2

1a) Parsing JSON from API (Python 2)

In Python 2 it was easier to implement JSON parsing:

import urllib2

json_data = urllib2.urlopen(url)
data = json.load(json_data)            # load() from file

.

1b) Parsing JSON from API (Python 3)

Python 3 dropped urllib2 and instead moved to a new standard for urllib:

import urllib.request

json_data = urllib.request.urlopen(url)

However the data returned from this function can't be processed in the same way as before. As the type is 'HTTPResponse' we must first read and decode it into something we can use.

# returns a utf-8 'bytes' object which still can't be processed
json_data = urllib.request.urlopen(url).read()

# decode into a string
str_json_data = json_data.decode('utf-8')

Now that we have a string, we can use the loads() function to process it into a valid JSON dictionary.

json_dict = json.loads(str_json_data)        # loads() from string

Notice the difference between the json.load() and json.loads() functions.

.

2) Parse multiple requests into list

Append to list as normal

data.append(json_dict)

.

3) Output to file

file = open("file.json", "w")
file.write(json.dumps(data))
file.close()

You can add indentation for formatting purposes:

file.write(json.dumps(data, indent=4, sort_keys=True)

Python 3 - Parse JSON from multiple API requests into a list and output to a file

2 个答案: