Question

我正在尝试以编程方式访问密歇根州肯特县的COVID-19测试数据。 CSV下载或JSON API端点会很棒，但我找不到这些端点。

这是在浏览器（Firefox）中访问数据的方式：

转到此处：https://www.michigan.gov/coronavirus/0,9753,7-406-98163_98173---,00.html
在页面上，单击“全面测试”标签
打开网络控制台（Ctrl + Shift + E）
在页面上的“选择县”下拉列表中，选择“肯特”
在“网络控制台”标签中，选择对https://wabi-us-gov-iowa-api.analysis.usgovcloudapi.net/public/reports/querydata?synchronous=true的最大（第四）后台POST请求
右键单击>在新选项卡中打开

偷看实际上在Firefox中加载数据的后台请求，这是我看到的：

请求方法

POST https://wabi-us-gov-iowa-api.analysis.usgovcloudapi.net/public/reports/querydata?synchronous=true

请求标头

Host: wabi-us-gov-iowa-api.analysis.usgovcloudapi.net
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/json;charset=UTF-8
Content-Length: 1160
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0, no-cache
Origin: https://wabi-us-gov-iowa-api.analysis.usgovcloudapi.net
Pragma: no-cache

请求正文

{"version":"1.0.0","queries":[{"Query":{"Commands":[{"SemanticQueryDataShapeCommand":{"Query":{"Version":2,"From":[{"Name":"t1","Entity":"Testing_Data"},{"Name":"c","Entity":"Cases by County"}],"Select":[{"Column":{"Expression":{"SourceRef":{"Source":"t1"}},"Property":"TestType"},"Name":"Testing_Data.TestType"},{"Column":{"Expression":{"SourceRef":{"Source":"t1"}},"Property":"MessageDate"},"Name":"Testing_Data.MessageDate"},{"Aggregation":{"Expression":{"Column":{"Expression":{"SourceRef":{"Source":"t1"}},"Property":"TestCount"}},"Function":0},"Name":"CountNonNull(Testing_Data.TestCount)"}],"Where":[{"Condition":{"In":{"Expressions":[{"Column":{"Expression":{"SourceRef":{"Source":"c"}},"Property":"COUNTY"}}],"Values":[[{"Literal":{"Value":"'Kent'"}}]]}}}]},"Binding":{"Primary":{"Groupings":[{"Projections":[1,2]}]},"Secondary":{"Groupings":[{"Projections":[0]}]},"DataReduction":{"DataVolume":4,"Primary":{"Sample":{}},"Secondary":{"Top":{}}},"Version":1}}}]},"QueryId":"","ApplicationContext":{"DatasetId":"3538771d-70f8-4399-9760-267975e37f65","Sources":[{"ReportId":"f489615d-c09e-43f9-b6bb-db2832eb0e0d"}]}}],"cancelQueries":[],"modelId":282246}

这是我天真地尝试使用Python请求重现该内容：

>>> import requests, json
>>> url = "https://wabi-us-gov-iowa-api.analysis.usgovcloudapi.net/public/reports/querydata?synchronous=true"
>>> origin = "https://wabi-us-gov-iowa-api.analysis.usgovcloudapi.net"
>>> host = "wabi-us-gov-iowa-api.analysis.usgovcloudapi.net"
>>> headers = {'Host': host, 'Origin': origin, 'DNT': '1'}
>>> body = """{"version":"1.0.0","queries":[{"Query":{"Commands":[{"SemanticQueryDataShapeCommand":{"Query":{"Version":2,"From":[{"Name":"t1","Entity":"Testing_Data"},{"Name":"c","Entity":"Cases by County"}],"Select":[{"Column":{"Expression":{"SourceRef":{"Source":"t1"}},"Property":"TestType"},"Name":"Testing_Data.TestType"},{"Column":{"Expression":{"SourceRef":{"Source":"t1"}},"Property":"MessageDate"},"Name":"Testing_Data.MessageDate"},{"Aggregation":{"Expression":{"Column":{"Expression":{"SourceRef":{"Source":"t1"}},"Property":"TestCount"}},"Function":0},"Name":"CountNonNull(Testing_Data.TestCount)"}],"Where":[{"Condition":{"In":{"Expressions":[{"Column":{"Expression":{"SourceRef":{"Source":"c"}},"Property":"COUNTY"}}],"Values":[[{"Literal":{"Value":"'Kent'"}}]]}}}]},"Binding":{"Primary":{"Groupings":[{"Projections":[1,2]}]},"Secondary":{"Groupings":[{"Projections":[0]}]},"DataReduction":{"DataVolume":4,"Primary":{"Sample":{}},"Secondary":{"Top":{}}},"Version":1}}}]},"QueryId":"","ApplicationContext":{"DatasetId":"3538771d-70f8-4399-9760-267975e37f65","Sources":[{"ReportId":"f489615d-c09e-43f9-b6bb-db2832eb0e0d"}]}}],"cancelQueries":[],"modelId":282246}"""
>>> requests.post(url, json=json.loads(body), headers=headers)
<Response [401]>

据我所知，不需要身份验证，但我怀疑还有其他限制，例如跨域访问。

我试图在Google上挖掘实际的政府API文档，但这有点麻烦。我找到此页面：

https://api.data.gov/list-of-apis/

但是其中有一半的链接404。欢迎任何帮助。谢谢！

Answer 1

我设法将成功的请求录音在一起。当我右键单击Firefox Web控制台中的“网络”选项卡请求时，发现“复制为cURL”选项有很大的帮助。这就是它给我的：

curl 'https://wabi-us-gov-iowa-api.analysis.usgovcloudapi.net/public/reports/querydata?synchronous=true' \
 -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0' \
 -H 'Accept: application/json, text/plain, */*' \
 -H 'Accept-Language: en-US,en;q=0.5' --compressed \
 -H 'Referer: https://app.powerbigov.us/view?r=eyJrIjoiMWNjYjU1YWQtNzFlMC00N2ZlLTg3NjItYmQxMWI4OWIwMGY1IiwidCI6ImQ1ZmI3MDg3LTM3NzctNDJhZC05NjZhLTg5MmVmNDcyMjVkMSJ9' \
 -H 'ActivityId: 62469790-1ee4-47e5-ae3e-183e05a7b299' \
 -H 'RequestId: ce1c2243-37ba-666e-831f-e4947a4ab1ac' 
 -H 'X-PowerBI-ResourceKey: 1ccb55ad-71e0-47fe-8762-bd11b89b00f5' \
 -H 'Content-Type: application/json;charset=UTF-8' \
 -H 'Origin: https://app.powerbigov.us' \
 -H 'DNT: 1' \
 -H 'Connection: keep-alive' \
 --data-raw $'{"version":"1.0.0", etc ...}'

这导致此脚本请求成功完成：

import requests
import json

url = 'https://wabi-us-gov-iowa-api.analysis.usgovcloudapi.net/public/reports/querydata'

# headers and body were simply copy-pasted from Firefox console.
headers = {
    'Host': 'wabi-us-gov-iowa-api.analysis.usgovcloudapi.net',
    'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv',
    'Accept': 'application/json, text/plain, */*',
    'Accept-Language': 'en-US,en;q=0.5',
    'Accept-Encoding': 'gzip, deflate, br',
    'Referer': 'https',
    'ActivityId': '62469790-1ee4-47e5-ae3e-183e05a7b299',
    'RequestId': 'ce1c2243-37ba-666e-831f-e4947a4ab1ac',
    'X-PowerBI-ResourceKey': '1ccb55ad-71e0-47fe-8762-bd11b89b00f5',
    'Content-Type': 'application/json;charset=UTF-8',
    'Origin': 'https',
    'DNT': '1',
    'Connection': 'keep-alive'
}

body = (
    '{"version":"1.0.0","queries":[{"Query":{"Commands":[{"SemanticQueryDataShapeCommand":'
    '{"Query":{"Version":2,"From":[{"Name":"t1","Entity":"Testing_Data"},{"Name":"c",'
    '"Entity":"Cases by County"}],"Select":[{"Column":{"Expression":{"SourceRef":'
    '{"Source":"t1"}},"Property":"TestType"},"Name":"Testing_Data.TestType"},{"Column":'
    '{"Expression":{"SourceRef":{"Source":"t1"}},"Property":"MessageDate"},"Name":'
    '"Testing_Data.MessageDate"},{"Aggregation":{"Expression":{"Column":{"Expression":'
    '{"SourceRef":{"Source":"t1"}},"Property":"TestCount"}},"Function":0},"Name":'
    '"CountNonNull(Testing_Data.TestCount)"}],"Where":[{"Condition":{"In":{"Expressions":'
    '[{"Column":{"Expression":{"SourceRef":{"Source":"c"}},"Property":"COUNTY"}}],'
    '"Values":[[{"Literal":{"Value":"\'Kent\'"}}]]}}}]},'
    '"Binding":{"Primary":{"Groupings":[{"Projections":[1,2]}]},"Secondary":{"Groupings":'
    '[{"Projections":[0]}]},"DataReduction":{"DataVolume":4,"Primary":{"Sample":{}},'
    '"Secondary":{"Top":{}}},"Version":1}}}]},"QueryId":"","ApplicationContext":'
    '{"DatasetId":"3538771d-70f8-4399-9760-267975e37f65","Sources":[{"ReportId":'
    '"f489615d-c09e-43f9-b6bb-db2832eb0e0d"}]}}],"cancelQueries":[],"modelId":282246}'
)

json_data = json.loads(body)
response = requests.post(url, json=json_data, headers=headers)
data = response.json()

我怀疑ID或密钥最终会失效，但是到目前为止，它们已经使用了24小时以上。

我仍然有兴趣查找有关访问此API端点的文档。

Answer 2

这个新的API（文档plansafe.xyz/api）正式提供了美国任何县的COVID-19数据。

下面是我对密歇根州肯特县当前数据进行的一些GET请求的图片。该API也可以用于历史数据（任何县的任何一天的COVID-19数据）。它也是免费的！

COVID-19 County Data for Kent County, Michigan

如果图像无法加载，我只是使用肯特和密歇根州作为文档指定的URL参数向端点发出了请求。我在Python上进行了测试，但是由于它是HTTP GET，因此可以与curl等配合使用。

如何以编程方式访问美国政府提供的县级COVID-19测试数据？

请求方法

请求标头

请求正文

2 个答案: