Question

问题-我正在尝试使用asyncio，async_timeout和aiohttp向不同的URL发出多个get请求。我只想在完成所有get请求之后或经过超时期限后（以先发生者为准）进行休息处理。如果所有任务都没有在指定的_DEFAULT_TIME_OUT时间内完成，那么无论完成什么获取请求，都只会继续执行它们。

在下面的代码片段中，即使我所有的任务都已完成，我也总是在等待_DEFAULT_TIME_OUT。如何显式解决超时问题。

async def get(self, session, url, attributes):
        timeout_period = self._DEFAULT_TIME_OUT)
        try:
            with async_timeout.timeout(timeout_period) as timeout:
                async with session.get(url) as response:
                    self.urls[url] = await response.content.read()
                    response.release()
                print("---{}---".format(url))
                raise asyncio.TimeoutError()
        except asyncio.TimeoutError:
            pass

async def http_request(self, even_loop):
    tasks = []
    async with aiohttp.ClientSession(loop=even_loop) as session:
        for url in all_urls:
            tasks.append(self.get(session, url, attributes))
        await asyncio.gather(*tasks)
        print("Do something else")

Answer 1

此示例说明如何使用async_timeout并获取有关任务的信息：

import aiohttp
import asyncio
import async_timeout


async def get(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.json()


async def main():
    timeout = 3.5

    tasks = [
        asyncio.create_task(get(f'http://httpbin.org/delay/{delay}'))
        for delay
        in range(1, 6)
    ]

    try:
        with async_timeout.timeout(timeout):
            await asyncio.gather(*tasks)
    except asyncio.TimeoutError:
        pass
    finally:
        for i, task in enumerate(tasks):
            if task.done() and not task.cancelled():
                print(f'Task is finished: {task.result()["url"]}.')
            else:
                print(f'Task hasn\'t been finished.')


asyncio.run(main())

结果：

Task is finished: http://httpbin.org/delay/1.
Task is finished: http://httpbin.org/delay/2.
Task is finished: http://httpbin.org/delay/3.
Task hasn't been finished.
Task hasn't been finished.

您可以使用timeout和delay变量来查看一切是否按预期进行：

timeout = 10

tasks = [
    asyncio.create_task(get(f'http://httpbin.org/delay/{delay}'))
    for delay
    in (1, 1, 1)
]

将在1秒后立即完成。完成所有任务。

使用asyncio和aiohttp在超时期限内进行多次获取请求

1 个答案: