Question

我有一个脚本可以使用asyncio下载文件。我面临的问题是，当出现异常时，它会在任务中被捕获，但任务会恢复并重新开始循环

在 downloader 中已实现分页循环，内循环 attemps 会在3次尝试后停止，并停止外循环 pagging 异常会出现并捕获到除外... ，此时协程应返回，但不是循环 attemps 再次开始，但这一次的值为2

仅当在相同的时间（当多个任务提交到循环中）运行多个任务（coroutne 下载器）时，此行为才会发生协程中有一种嫌疑行为

async def downloader(session, url, spe_params, path, name, writer,
                     max_features=5000, sleeping=5):
    try:
        pagging = True
        while pagging:
            attemps = 3
            while attemps:
                # attemps reduced for looping
                try:
                    async with session.get(url, params=p) as res:
                        if res.status != 200:
                            raise aiohttp.ClientError
                        res_json = await res.json()

                except (aiohttp.ClientError, asyncio.TimeoutError,
                        json.JSONDecodeError) as e:
                    message = 'Error fetching {0}'.format(str(e))
                    attemps -= 1
                    await asyncio.sleep(sleeping)
                else:
                    if res_json['totalFeatures'] == number_returned:
                        pagging = False
                        break
                    # Here the logic for pagination

            if not attemps:
                # attemps have been consumed
                # Pagination stops and raise exception
                raise DownloadError(message)

    except DownloadError as e:
        # Exception is catched. The coroutine should end but instead of that
        # the loop "attemps" start again with the original value - 1

    else:
        # Json written to file

async def main():
    loop_tasks = []
    for id in field_id:
        task = loop.create_task(downloader(session, ... )

    done, pending = await asyncio.wait(loop_tasks,
                                               return_when='ALL_COMPLETED')

在异步协程中捕获异常

0 个答案: