Question

问题

使用Python的result = yield from select(asyncio.sleep(1), queue.get())模块，如何从多个协同程序中选择第一个结果？

实施例

我可能希望在等待队列时实现超时：

all

Analagous Operations

这类似于Go's select或Clojure's core.async.alt!。它与asyncio.gather的相反（收集类似于any，select就像let socketConn = Connection(); socketConn.connect();。）

Answer 1

简单的解决方案，使用asyncio.wait及其FIRST_COMPLETED参数：

import asyncio

async def something_to_wait():
    await asyncio.sleep(1)
    return "something_to_wait"

async def something_else_to_wait():
    await asyncio.sleep(2)
    return "something_else_to_wait"


async def wait_first():
    done, pending = await asyncio.wait(
        [something_to_wait(), something_else_to_wait()],
        return_when=asyncio.FIRST_COMPLETED)
    print("done", done)
    print("pending", pending)

asyncio.get_event_loop().run_until_complete(wait_first())

给出：

done {<Task finished coro=<something_to_wait() done, defined at stack.py:3> result='something_to_wait'>}
pending {<Task pending coro=<something_else_to_wait() running at stack.py:8> wait_for=<Future pending cb=[Task._wakeup()]>>}
Task was destroyed but it is pending!
task: <Task pending coro=<something_else_to_wait() running at stack.py:8> wait_for=<Future pending cb=[Task._wakeup()]>>

Answer 2

您可以同时使用asyncio.wait和asyncio.as_completed：

来实现此目的

import asyncio

@asyncio.coroutine
def ok():
    yield from asyncio.sleep(1)
    return 5

@asyncio.coroutine
def select1(*futures, loop=None):
    if loop is None:
        loop = asyncio.get_event_loop()
    return (yield from next(asyncio.as_completed(futures)))

@asyncio.coroutine
def select2(*futures, loop=None):
    if loop is None:
        loop = asyncio.get_event_loop()
    done, running = yield from asyncio.wait(futures,
                                            return_when=asyncio.FIRST_COMPLETED)
    result = done.pop()
    return result.result()

@asyncio.coroutine
def example():
    queue = asyncio.Queue()
    result = yield from select1(ok(), queue.get())
    print('got {}'.format(result))
    result = yield from select2(queue.get(), ok())
    print('got {}'.format(result))

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(example())

输出：

got 5
got 5
Task was destroyed but it is pending!
task: <Task pending coro=<get() done, defined at /usr/lib/python3.4/asyncio/queues.py:170> wait_for=<Future pending cb=[Task._wakeup()]> cb=[as_completed.<locals>._on_completion() at /usr/lib/python3.4/asyncio/tasks.py:463]>
Task was destroyed but it is pending!
task: <Task pending coro=<get() done, defined at /usr/lib/python3.4/asyncio/queues.py:170> wait_for=<Future pending cb=[Task._wakeup()]>>

两个实现都返回第一个完成的Future产生的值，但您可以轻松调整它以返回Future本身。请注意，因为传递给每个Future实现的其他select永远不会产生，所以当进程退出时会发出警告。

Answer 3

在想要对任务应用超时的情况下，有一个标准的库函数可以完成这个：asyncio.wait_for()。您的示例可以这样写：

try:
  result = await asyncio.wait_for(queue.get(), timeout=1)
except asyncio.TimeoutError:
  # This block will execute if queue.get() takes more than 1s.
  result = ...

但这仅适用于超时的特定情况。这里的另外两个答案概括为任意一组任务，但这些答案都没有显示如何清理先未完成的任务。这是导致输出中的“任务被销毁但正在等待”消息的原因。在实践中，您应该对那些待处理任务执行某些操作。根据您的示例，我假设您不关心其他任务的结果。这是一个wait_first()函数的示例，它返回第一个已完成任务的值并取消剩余的任务。

import asyncio, random

async def foo(x):
    r = random.random()
    print('foo({:d}) sleeping for {:0.3f}'.format(x, r))
    await asyncio.sleep(r)
    print('foo({:d}) done'.format(x))
    return x

async def wait_first(*futures):
    ''' Return the result of the first future to finish. Cancel the remaining
    futures. '''
    done, pending = await asyncio.wait(futures,
        return_when=asyncio.FIRST_COMPLETED)
    gather = asyncio.gather(*pending)
    gather.cancel()
    try:
        await gather
    except asyncio.CancelledError:
        pass
    return done.pop().result()

async def main():
    result = await wait_first(foo(1), foo(2))
    print('the result is {}'.format(result))

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()

运行此示例：

# export PYTHONASYNCIODEBUG=1
# python3 test.py
foo(1) sleeping for 0.381
foo(2) sleeping for 0.279
foo(2) done
the result is 2
# python3 test.py
foo(1) sleeping for 0.048
foo(2) sleeping for 0.515
foo(1) done
the result is 1
# python3 test.py
foo(1) sleeping for 0.396
foo(2) sleeping for 0.188
foo(2) done
the result is 2

没有关于待处理任务的错误消息，因为每个待处理任务都已正确清理。

在实践中，您可能希望wait_first()返回未来，而不是未来的结果，否则尝试找出未来的结果将会非常困惑。但是在这里的例子中，我回到了未来的结果，因为它看起来更清洁。

Answer 4

这里有一个更强大的解决方案，它基于处理以下内容的早期示例：

使用递归返回第一个非空结果（前面的例子将返回第一个结果，无论是空还是非空）
即使另一个任务引发异常，也返回第一个非空结果
如果只返回非空结果并引发异常，则引发最后一个异常
处理同时完成的多个任务 - 在实践中这种情况很少见，但它可能会出现在单元测试中，假异步任务会立即完成。

请注意，由于使用了赋值运算符，此示例需要 Python 3.8。

async def wait_first(*tasks):
    """Return the result of first async task to complete with a non-null result"""
    # Get first completed task(s)
    done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)

    # Tasks MAY complete at same time e.g. in unit tests :)
    # Coalesce the first result if present
    for task in done:
        exception = task.exception()
        if exception is None and (result := task.result()):
            break
        else:
            result = None
        
    # Gather remaining tasks without raising exceptions
    gather = asyncio.gather(*pending, return_exceptions=True)

    # Cancel remaining tasks if result is non-null otherwise await next pending tasks
    if result:
        gather.cancel()
    elif pending:
        result = await wait_first(*pending)
    
    # Await remaining tasks to ensure they are cancelled
    try:
        await gather
    except asyncio.CancelledError:
        pass
    
    # Return result or raise last exception if no result was returned
    if exception and result is None:
        raise exception
    else:
        return result

从asyncio中的两个协同程序中选择第一个结果

问题

实施例

Analagous Operations

4 个答案: