在中断事件循环后清理的正确方法是什么？

时间：2015-06-10 19:30:46

标签： python python-3.4 python-asyncio

我有一个事件循环，它运行一些协同例程作为命令行工具的一部分。用户可以使用通常的 Ctrl + C 中断工具，此时我想在中断的事件循环后正确清理。

这是我尝试过的。

markdown.pl

运行此命令并按 Ctrl + C 会产生：

import asyncio


@asyncio.coroutine
def shleepy_time(seconds):
    print("Shleeping for {s} seconds...".format(s=seconds))
    yield from asyncio.sleep(seconds)


if __name__ == '__main__':
    loop = asyncio.get_event_loop()

    # Side note: Apparently, async() will be deprecated in 3.4.4.
    # See: https://docs.python.org/3.4/library/asyncio-task.html#asyncio.async
    tasks = [
        asyncio.async(shleepy_time(seconds=5)),
        asyncio.async(shleepy_time(seconds=10))
    ]

    try:
        loop.run_until_complete(asyncio.gather(*tasks))
    except KeyboardInterrupt as e:
        print("Caught keyboard interrupt. Canceling tasks...")

        # This doesn't seem to be the correct solution.
        for t in tasks:
            t.cancel()
    finally:
        loop.close()

显然，我没有正确清理。我想也许在任务上调用$ python3 asyncio-keyboardinterrupt-example.py Shleeping for 5 seconds... Shleeping for 10 seconds... ^CCaught keyboard interrupt. Canceling tasks... Task was destroyed but it is pending! task: <Task pending coro=<shleepy_time() running at asyncio-keyboardinterrupt-example.py:7> wait_for=<Future cancelled> cb=[gather.<locals>._done_callback(1)() at /usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py:587]> Task was destroyed but it is pending! task: <Task pending coro=<shleepy_time() running at asyncio-keyboardinterrupt-example.py:7> wait_for=<Future cancelled> cb=[gather.<locals>._done_callback(0)() at /usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py:587]> 就可以了。

在中断的事件循环后清理的正确方法是什么？

5 个答案:

答案 0 :(得分：39)

当您按CTRL + C时，事件循环会停止，因此您对t.cancel()的调用实际上不会生效。对于要取消的任务，您需要重新启动循环。

以下是您可以处理的方法：

import asyncio

@asyncio.coroutine
def shleepy_time(seconds):
    print("Shleeping for {s} seconds...".format(s=seconds))
    yield from asyncio.sleep(seconds)


if __name__ == '__main__':
    loop = asyncio.get_event_loop()

    # Side note: Apparently, async() will be deprecated in 3.4.4.
    # See: https://docs.python.org/3.4/library/asyncio-task.html#asyncio.async
    tasks = asyncio.gather(
        asyncio.async(shleepy_time(seconds=5)),
        asyncio.async(shleepy_time(seconds=10))
    )

    try:
        loop.run_until_complete(tasks)
    except KeyboardInterrupt as e:
        print("Caught keyboard interrupt. Canceling tasks...")
        tasks.cancel()
        loop.run_forever()
        tasks.exception()
    finally:
        loop.close()

收到KeyboardInterrupt后，我们会调用tasks.cancel()，然后再次启动loop。 run_forever会在tasks取消后立即退出（请注意，取消Future返回的asyncio.gather也会取消其中Futures内的所有loop.run_until_complete，因为中断的done_callback调用添加了tasks到tasks来停止循环。因此，当我们取消tasks.exception时，该回调将触发，并且循环停止。此时我们调用_GatheringFuture，以避免收到有关未从 console.log( "li index =", li.index() ); 获取异常的警告。

答案 1 :(得分：12)

已针对Python 3.6 + 进行了更新：添加对loop.shutdown_asyncgens的调用，以避免因未完全使用的异步生成器而导致内存泄漏。此外，现在使用asyncio.new_event_loop而不是asyncio.get_event_loop来确保最终loop.close调用不会干扰循环的其他可能用途。

以下解决方案受到其他一些答案的启发，几乎适用于所有情况，并且不依赖于您手动跟踪需要清理的任务 Ctrl + C ：

loop = asyncio.new_event_loop()
try:
    # Here `amain(loop)` is the core coroutine that may spawn any
    # number of tasks
    sys.exit(loop.run_until_complete(amain(loop)))
except KeyboardInterrupt:
    # Optionally show a message if the shutdown may take a while
    print("Attempting graceful shutdown, press Ctrl+C again to exit…", flush=True)

    # Do not show `asyncio.CancelledError` exceptions during shutdown
    # (a lot of these may be generated, skip this if you prefer to see them)
    def shutdown_exception_handler(loop, context):
        if "exception" not in context \
        or not isinstance(context["exception"], asyncio.CancelledError):
            loop.default_exception_handler(context)
    loop.set_exception_handler(shutdown_exception_handler)

    # Handle shutdown gracefully by waiting for all tasks to be cancelled
    tasks = asyncio.gather(*asyncio.Task.all_tasks(loop=loop), loop=loop, return_exceptions=True)
    tasks.add_done_callback(lambda t: loop.stop())
    tasks.cancel()

    # Keep the event loop running until it is either destroyed or all
    # tasks have really terminated
    while not tasks.done() and not loop.is_closed():
        loop.run_forever()
finally:
    if hasattr(loop, "shutdown_asyncgens"):  # This check is only needed for Python 3.5 and below
        loop.run_until_complete(loop.shutdown_asyncgens())
    loop.close()

上面的代码将使用asyncio.Task.all_tasks从事件循环中获取所有当前任务，并使用asyncio.gather将它们放在一个组合的未来中。然后，使用未来的.cancel()方法取消该将来的所有任务（这些任务都是当前正在运行的任务）。 return_exceptions=True然后确保存储所有收到的asyncio.CancelledError例外，而不是导致未来出错。

上面的代码也将覆盖默认的异常处理程序，以防止记录生成的asyncio.CancelledError异常。

答案 2 :(得分：3)

在 Python 3.7 + 中，建议您使用asyncio.run启动异步主函数。

Hostname = "YOUR_SERVER.database.windows.net" Database = "YOUR_DB" port = 1433 UN = 'YOUR_USERNAME' PW = 'YOUR_PASSWORD' Url = "jdbc:sqlserver://{0}:{1};database={2};user={3};password= {4}".format(Hostname, Port, Database, UN, PW) df.write.jdbc(Url, "schema.table", mode="append") 将负责为您的程序创建事件循环，并确保在主函数退出时（包括由于asyncio.run异常）关​​闭事件循环并清除所有任务。

它大致类似于以下内容（请参见asyncio/runners.py）：

KeyboardInterrupt

答案 3 :(得分：2)

除非您在Windows上，否则为SIGINT（以及SIGTERM设置基于事件循环的信号处理程序，以便您可以将其作为服务运行）。在这些处理程序中，您可以立即退出事件循环，也可以启动某种清理序列并稍后退出。

官方Python文档中的示例：https://docs.python.org/3.4/library/asyncio-eventloop.html#set-signal-handlers-for-sigint-and-sigterm

答案 4 :(得分：0)

使用 signal 模块在 asyncio.Event 信号（Ctrl + C）上设置 signal.SIGINT 可以是让所有异步代码自然停止的干净方式。这一点尤其重要，因为一些库，例如 aiohttp need a chance to be run cleanup tasks before the event loop closes.

这是一个使用 aiohttp 库的示例。有一个asyncio.sleep(5)来防止连接返回到池中，给用户一个ctrl+c的机会，模拟一个KeyboardInterrupt异常

示例代码：

import logging
import asyncio
import signal
import random

import aiohttp

logging.basicConfig(level="INFO", format="%(asctime)s %(threadName)-10s %(name)-10s %(levelname)-8s: %(message)s")
logger = logging.getLogger("root")

stop_event = asyncio.Event()

async def get_json(aiohttp_session):

    logger.info("making http request")

    params = {"value": random.randint(0,1000) }
    async with aiohttp_session.get(f'https://httpbin.org/get', params=params) as response:

        # async with response:
        j = await response.json()
        logger.info("get data: `%s`", j["args"])
        await asyncio.sleep(5)

async def run():

    while not stop_event.is_set():
        async with aiohttp.ClientSession() as aiohttp_session:

            await get_json(aiohttp_session)

    logger.info("stop event was set, sleeping to let aiohttp close it's connections")
    await asyncio.sleep(0.1)
    logger.info("sleep finished, returning")


def inner_ctrl_c_signal_handler(sig, frame):
    '''
    function that gets called when the user issues a
    keyboard interrupt (ctrl+c)
    '''

    logger.info("SIGINT caught!")
    stop_event.set()

# experiment with commenting out this line and ctrl+c-ing the script
# to see how you get an "event loop is closed" error
signal.signal(signal.SIGINT, inner_ctrl_c_signal_handler)

asyncio.run(run())

没有 signal.signal 调用：

> python C:\Users\mark\Temp\test_aiohttp.py
2021-03-06 22:21:08,684 MainThread root       INFO    : making http request
2021-03-06 22:21:09,132 MainThread root       INFO    : get data: `{'value': '500'}`
Traceback (most recent call last):
  File "C:\Users\auror\Temp\test_aiohttp.py", line 52, in <module>
    asyncio.run(run())
  File "c:\python39\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "c:\python39\lib\asyncio\base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "c:\python39\lib\asyncio\windows_events.py", line 316, in run_forever
    super().run_forever()
  File "c:\python39\lib\asyncio\base_events.py", line 596, in run_forever
    self._run_once()
  File "c:\python39\lib\asyncio\base_events.py", line 1854, in _run_once
    event_list = self._selector.select(timeout)
  File "c:\python39\lib\asyncio\windows_events.py", line 434, in select
    self._poll(timeout)
  File "c:\python39\lib\asyncio\windows_events.py", line 783, in _poll
    status = _overlapped.GetQueuedCompletionStatus(self._iocp, ms)
KeyboardInterrupt
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001CFFD75BB80>
Traceback (most recent call last):
  File "c:\python39\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "c:\python39\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "c:\python39\lib\asyncio\base_events.py", line 746, in call_soon
    self._check_closed()
  File "c:\python39\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

与它：


> python C:\Users\mark\Temp\test_aiohttp.py
2021-03-06 22:20:29,656 MainThread root       INFO    : making http request
2021-03-06 22:20:30,106 MainThread root       INFO    : get data: `{'value': '367'}`
2021-03-06 22:20:35,122 MainThread root       INFO    : making http request
2021-03-06 22:20:35,863 MainThread root       INFO    : get data: `{'value': '489'}`
2021-03-06 22:20:38,695 MainThread root       INFO    : SIGINT caught!
2021-03-06 22:20:40,867 MainThread root       INFO    : stop event was set, sleeping to let aiohttp close it's connections
2021-03-06 22:20:40,962 MainThread root       INFO    : sleep finished, returning
相关问题
最新问题