Question

我正在尝试处理异步HTTP请求。我从另一个模块调用async_provider（）函数，并使用生成的response.text（）执行后续任务。它仅在所有请求成功后才起作用。但是我无法处理失败请求的任何异常（无论异常原因如何）。谢谢您的帮助。这是代码的相关部分：

import asyncio
import aiohttp

# i call this function from another module
def async_provider():
    list_a, list_b = asyncio.run(main())
    return list_a, list_b


async def fetch(session, url):
    # session.post request cases
    if url == "http://...1":
        referer = "http://...referer"
        user_agent = (
            "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) "
            "AppleWebKit/605.1.15 (KHTML, like Gecko) "
            "Version/12.1 Safari/605.1.15"
        )
        payload = {'key1': 'value1', 'key2': 'value2'}
        async with session.post(
            url, data=payload, headers={"Referer": referer, "User-Agent": user_agent}
        ) as response:
            if response.status != 200:
                response.raise_for_status()
            return await response.text()

    # session.get request cases
    else:
        async with session.get(url) as response:
            if response.status != 200:
                response.raise_for_status()
            return await response.text()


async def fetch_all(session, urls):
    results = await asyncio.gather(
        *[asyncio.create_task(fetch(session, url)) for url in urls]
    )
    return results


async def main():
    urls = ["http://...1", "http://...2", "http://...3"]
    async with aiohttp.ClientSession() as session:
        response_text_1, response_text_2, response_text_3 = await fetch_all(
            session, urls
        )
    # some task with response text

任何异常都会中断所有请求

Answer 1

选中"return_exceptions" flag on gather。

results = await asyncio.gather(
    *[asyncio.create_task(fetch(session, url)) for url in urls],
    return_exceptions=True
)

它将返回您已完成任务的列表。然后，您可以使用他们的Task.result()或 Task.exception()种引发或检查是否存在异常的方法。

Python 3.x中异步HTTP请求的异常处理

1 个答案: