使用python生成器处理url分页

Question

目前，我仅从服务器获取第一页，json的一部分是

{"status":"success","count":100,"total":22188,"next":"https://pimber.ly/api/v2/products/?sinceId=5981e16fcde47c0854dc540b","previous":"https://pimber.ly/api/v2/products/?maxId=5981e01dcde47c0854dc4afd","sinceId":"5981e01dcde47c0854dc4afd","maxId":"5981e16fcde47c0854dc540b","data":[.....]}

函数是：

_fetch_data = response.json()
while _fetch_data['next'] is not None:
    response = requests.get(
        url=API_DOMAIN',
        headers=headers
    )
    _page_data = response.json()['data']
    for _data in _page_data:
        yield _data

该函数的当前状态仅在处理第一页，它将永远做下去，所以我该如何修复该函数以检查next，以便可以获取total数据？

Answer 1

我想应该是

_fetch_data = response.json()
while _fetch_data['next'] is not None:
    response = requests.get(_fetch_data['next'], headers=headers)
    _fetch_data = response.json()
    for _data in fetch_data['data']:
        yield _data