问题:在asyncio.as_completed产生结果后,您如何获得对原始任务的引用?
除了Python之外,基本上和C#问题一样:Getting reference to original Task after ordering Tasks by completion?
示例问题:
# Takes a list of WebClient objects,
# calls each one simultaneously,
# and yields the results immediately as they arrive
# to a synchronous caller.
def yieldThingsAsTheyArrive(webClients):
tasks = []
for webClient in webClients:
# This is what we want to get a reference to later:
task = webClient.fetch_thing() # start long-running request asynchronously
tasks.append(task)
loop = asyncio.get_event_loop()
for future in asyncio.as_completed(tasks):
thing = loop.run_until_complete(future) # since our caller is synchronous, wait until the task completes so we can yield the final result instead of a future
thing.originalWebClient = ??? # This is where we need a reference to the original webClient
yield thing
答案 0 :(得分:3)
as_completed
具体是因为它既不会产生像asyncio.wait
这样的未来,也不会产生像asyncio.gather
这样的结果。相反,它会产生您需要等待的协程(以您喜欢的任何方式)以完成顺序获得结果。它不会产生你传递给它的期货,因为那时它还不知道下一个通过的期货将会完成。
您可以通过在另一个未来包装任务来关联任意数据,其结果是任务对象(您已将数据附加到该对象)。这基本上等同于C# code所做的,只是没有静态打字仪式。从this answer进行设置,可运行的示例如下所示:
import asyncio
async def first():
await asyncio.sleep(5)
return 'first'
async def second():
await asyncio.sleep(1)
return 'second'
async def third():
await asyncio.sleep(3)
return 'third'
def ordinary_generator():
loop = asyncio.get_event_loop()
wrappers = []
for idx, coro in enumerate((first(), second(), third())):
task = loop.create_task(coro)
task.idx = idx + 1
# Wrap the task in a future that completes when the
# task does, but whose result is the task object itself.
wrapper = loop.create_future()
task.add_done_callback(wrapper.set_result)
wrappers.append(wrapper)
for x in asyncio.as_completed(wrappers):
# yield completed tasks
yield loop.run_until_complete(x)
for task in ordinary_generator():
print(task.result(), task.idx)
我建议的另一个选项是使用调用as_completed
的循环替换asyncio.wait(return_when=FIRST_COMPLETED)
上的迭代。这也将提供期货,因为它们是完整的,但不需要额外的包装,并导致稍微更惯用的asyncio代码。我们在每个协程上调用ensure_future
将其转换为将来,将数据附加到其中,然后将其传递给asyncio.wait()
。由于wait
返回那些相同的未来,所附的数据就在它们上面。
def ordinary_generator():
loop = asyncio.get_event_loop()
pending = []
for idx, coro in enumerate((first(), second(), third())):
task = loop.create_task(coro)
task.idx = idx + 1
pending.append(task)
while pending:
done, pending = loop.run_until_complete(asyncio.wait(
pending, return_when=asyncio.FIRST_COMPLETED))
for task in done:
yield task