等待条件变量超时:未及时重新获得锁

时间:2019-12-13 01:16:45

标签: python python-asyncio

我有一个名为asyncio.Condition的{​​{1}}。我希望等待,但只能等待很久才放弃。由于cond不会超时,因此无法直接完成。 The docs指出应使用asyncio.Condition.wait来包装并提供超时:

  

asyncio.wait_for()函数可用于在超时后取消任务。

因此,我们得出以下解决方案:

asyncio.wait_for

现在假定async def coro(): print("Taking lock...") async with cond: print("Lock acquired.") print("Waiting!") await asyncio.wait_for(cond.wait(), timeout=999) print("Was notified!") print("Lock released.") 本身在运行五秒钟后被取消。这会将coro抛出到CancelledError中,从而在重新引发错误之前取消wait_for。错误然后传播到cond.wait,由于coro块,该错误隐式尝试释放async with中的锁。但是,该锁当前未处于锁定状态。 cond已被取消,但没有机会处理该取消并重新获得锁。因此,我们得到了一个丑陋的异常,如下所示:

cond.wait

换句话说,在处理Taking lock... Lock acquired. Waiting! ERROR:asyncio:Task exception was never retrieved future: <Task finished coro=<coro() done, defined at [REDACTED]> exception=RuntimeError('Lock is not acquired.',)> Traceback (most recent call last): [REDACTED], in coro await asyncio.wait_for(cond.wait(), timeout=999) [REDACTED], in wait_for yield from waiter concurrent.futures._base.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): [REDACTED], in coro print("Was notified!") [REDACTED], in coro res = func(*args, **kw) [REDACTED], in __aexit__ self.release() [REDACTED], in release raise RuntimeError('Lock is not acquired.') RuntimeError: Lock is not acquired. 时,CancelledError从试图释放未持有的锁中引出coro。 stacktrace显示RuntimeError行的原因是因为这是有问题的print("Was notified!")块的最后一行。


这感觉我无法解决。我开始怀疑这是库本身的错误。但是,我想不出任何办法来避免该问题或创建解决方法,因此任何想法都将不胜感激。

在编写此问题并进行进一步调查的同时,我在Python错误跟踪器上遇到了类似的问题,最终检查了async with源代码,并确定实际上这是{{1}中的错误}本身。

我已将其提交给问题跟踪者here,以解决同样的问题,并使用我创建的解决方法回答了自己的问题。


编辑:按照ParkerD的要求,这是产生上述问题的完整可运行示例:

编辑2:的示例已更新,可以使用Python 3.7+的新asyncioasyncio功能

asyncio.run

1 个答案:

答案 0 :(得分:0)

正如问题末尾所述,我已确定问题实际上是库中的错误。我将重申该错误的问题跟踪器为here,并介绍我的解决方法。

以下函数基于asyncio.create_task本身(源here),并且是该函数的一个版本,专门用于条件等待,并另外保证取消它是安全的。

呼叫import asyncio async def coro(): cond = asyncio.Condition() print("Taking lock...") async with cond: print("Lock acquired.") print("Waiting!") await asyncio.wait_for(cond.wait(), timeout=999) print("Was notified!") print("Lock released.") async def cancel_after_5(c): task = asyncio.create_task(c) await asyncio.sleep(5) task.cancel() await asyncio.wait([task]) asyncio.run(cancel_after_5(coro())) 大致等效于wait_for

wait_on_condition_with_timeout(cond, timeout)

关键部分是,如果发生超时或取消,该方法将在重新引发异常之前等待条件重新获取锁:

asyncio.wait_for(cond.wait(), timeout)

我已经在Python 3.6.9上进行了测试,它可以完美运行。 3.7和3.8中也存在相同的错误,因此我认为它对于那些版本也很有用。如果您想知道何时修复错误,请检查上面的问题跟踪器。如果您想为async def wait_on_condition_with_timeout(condition: asyncio.Condition, timeout: float) -> bool: loop = asyncio.get_event_loop() # Create a future that will be triggered by either completion or timeout. waiter = loop.create_future() # Callback to trigger the future. The varargs are there to consume and void any arguments passed. # This allows the same callback to be used in loop.call_later and wait_task.add_done_callback, # which automatically passes the finished future in. def release_waiter(*_): if not waiter.done(): waiter.set_result(None) # Set up the timeout timeout_handle = loop.call_later(timeout, release_waiter) # Launch the wait task wait_task = loop.create_task(condition.wait()) wait_task.add_done_callback(release_waiter) try: await waiter # Returns on wait complete or timeout if wait_task.done(): return True else: raise asyncio.TimeoutError() except (asyncio.TimeoutError, asyncio.CancelledError): # If timeout or cancellation occur, clean up, cancel the wait, let it handle the cancellation, # then re-raise. wait_task.remove_done_callback(release_waiter) wait_task.cancel() await asyncio.wait([wait_task]) raise finally: timeout_handle.cancel() 以外的版本提供版本,则更改参数和except (asyncio.TimeoutError, asyncio.CancelledError): # If timeout or cancellation occur, clean up, cancel the wait, let it handle the cancellation, # then re-raise. wait_task.remove_done_callback(release_waiter) wait_task.cancel() await asyncio.wait([wait_task]) # This line is missing from the real wait_for raise 行应该很简单。