在python3中合并异步迭代

时间:2018-02-16 16:17:46

标签: python-3.x reactive-programming python-asyncio

在python3中合并异步迭代器有没有好方法或者支持良好的库?

所需的行为与在reactivex中合并observable的行为基本相同。

也就是说,在正常情况下,如果我正在合并两个异步迭代器,我希望生成的异步迭代器按时间顺序生成结果。其中一个迭代器中的错误应该使合并的迭代器脱轨。

Merging Observables

(资料来源:http://reactivex.io/documentation/operators/merge.html

这是我最好的尝试,但似乎可能有一个标准的解决方案:

async def drain(stream, q, sentinal=None):
    try:
        async for item in stream:
            await q.put(item)
        if sentinal:
            await q.put(sentinal)
    except BaseException as e:
        await q.put(e)


async def merge(*streams):

    q = asyncio.Queue()
    sentinal = namedtuple("QueueClosed", ["truthy"])(True)

    futures = {
        asyncio.ensure_future(drain(stream, q, sentinal)) for stream in streams
    }

    remaining = len(streams)
    while remaining > 0:
        result = await q.get()
        if result is sentinal:
            remaining -= 1
            continue
        if isinstance(result, BaseException):
            raise result
        yield result


if __name__ == "__main__":

    # Example: Should print:
    #   1
    #   2
    #   3
    #   4

    loop = asyncio.get_event_loop()

    async def gen():
        yield 1
        await asyncio.sleep(1.5)
        yield 3

    async def gen2():
        await asyncio.sleep(1)
        yield 2
        await asyncio.sleep(1)
        yield 4

    async def go():
        async for x in merge(gen(), gen2()):
            print(x)

    loop.run_until_complete(go())

1 个答案:

答案 0 :(得分:4)

您可以使用aiostream.stream.merge

from aiostream import stream

async def go():
    async for x in stream.merge(gen(), gen2()):
        print(x)

documentation和此answer中的更多示例。