TL; DR
这确实是Motor 1.2.0中的一个错误,该错误由A. Jesse Jiryu Davis迅速修复,并且可以在1.2.1或更高版本的驱动程序中使用。
原始问题
我在Python 3上编写了一个程序,用于监视MongoDB集合的更改,使用其新的Change Stream功能。这是MCVE:
from asyncio import get_event_loop, CancelledError
from contextlib import suppress
from motor.motor_asyncio import AsyncIOMotorClient
async def watch(collection):
async with collection.watch([]) as stream:
async for change in stream:
print(change)
async def cleanup():
task.cancel()
with suppress(CancelledError):
await task
if __name__ == '__main__':
conn = AsyncIOMotorClient()
loop = get_event_loop()
task = loop.create_task(watch(conn.database.collection)) # Replace with a real collection.
try:
loop.run_forever()
except KeyboardInterrupt:
pass
finally:
loop.run_until_complete(cleanup())
loop.shutdown_asyncgens()
loop.close()
当我用CRTL + C杀死程序时,它会引发三种不同的例外。
^Cexception calling callback for <Future at 0x102efea58 state=finished raised InvalidStateError>
Traceback (most recent call last):
File "/Users/viotti/motor/lib/python3.6/site-packages/motor/core.py", line 1259, in _next
change = self.delegate.next()
File "/Users/viotti/motor/lib/python3.6/site-packages/pymongo/change_stream.py", line 79, in next
change = self._cursor.next()
File "/Users/viotti/motor/lib/python3.6/site-packages/pymongo/command_cursor.py", line 292, in next
raise StopIteration
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/viotti/motor/lib/python3.6/site-packages/motor/core.py", line 1264, in _next
future.set_exception(StopAsyncIteration())
asyncio.base_futures.InvalidStateError: invalid state
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/_base.py", line 324, in _invoke_callbacks
callback(self)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/futures.py", line 414, in _call_set_state
dest_loop.call_soon_threadsafe(_set_state, destination, source)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 620, in call_soon_threadsafe
self._check_closed()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 357, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
有没有办法让这个程序默默关闭?
我在macOS Sierra上使用Python 3.6.4,Motor 1.2和pymongo 3.6.0进行测试。
答案 0 :(得分:2)
我认为您的代码是正确的,motor
方面存在问题。
在调查时我发现了两个问题:
exception calling callback for <Future
错误。它似乎与异步生成器或流无关,而与任何motor
用法有关。AgnosticChangeStream
异步迭代机制(_next function)在没有考虑取消的情况下编写。尝试将取消的未来设置为InvalidStateError
。此代码演示了两个问题和可能的解决方法:
import types
import asyncio
from contextlib import suppress
from motor.motor_asyncio import AsyncIOMotorClient
async def test():
while True:
await asyncio.sleep(0.1)
async def cleanup(task):
task.cancel()
with suppress(asyncio.CancelledError):
await task
def _next(self, future):
try:
if not self.delegate:
self.delegate = self._collection.delegate.watch(**self._kwargs)
change = self.delegate.next()
self._framework.call_soon(self.get_io_loop(),
future.set_result,
change)
except StopIteration:
future.set_exception(StopAsyncIteration())
except Exception as exc:
# CASE 2:
# Cancellation of async iteration (and future with it) happens immediately
# and trying to set exception to cancelled future leads to InvalidStateError,
# we should prevent it:
if future.cancelled():
return
future.set_exception(exc)
async def watch(collection):
async with collection.watch([]) as stream:
# Patch stream to achieve CASE 2:
stream._next = types.MethodType(_next, stream)
async for change in stream:
print(change)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
tmp = asyncio.ensure_future(test()) # Way to receive KeyboardInterrupt immediately.
client = AsyncIOMotorClient()
collection = client.test_database.test_collection
task = asyncio.ensure_future(watch(collection))
try:
loop.run_forever()
except KeyboardInterrupt:
print('KeyboardInterrupt')
finally:
loop.run_until_complete(cleanup(tmp))
loop.run_until_complete(cleanup(task))
# CASE 1:
# Looks like propagating KeyboardInterrupt doesn't affect motor's try
# to establish connection to db and I didn't find a way to stop this manually.
# We should keep event loop alive until we receive ServerSelectionTimeoutError
# and motor would be able to execute it's asyncio callbacks:
loop.run_until_complete(asyncio.sleep(client.server_selection_timeout))
loop.shutdown_asyncgens()
loop.close()
由于添加了修复,它完成时没有警告/异常(至少在我的机器上)。
我不建议你使用上面的黑客!这只是为了展示问题所在和可能的解决方案。我不确定它能做好一切。
相反,我建议你create issue at motor user group / Jira附加你的片段,可能是我的答案,等到bug修好。