async / await:从异步方法连续流式传输数据

时间:2018-01-19 14:51:54

标签: python python-3.x asynchronous python-asyncio

我使用python 3.5将数据从一个方法异步返回到另一个方法,如下所示:

async def A():
    # Need to get data here from B continuously
    val = await B()

async def B():
    # Need to get data here from C continuously as they get generated inside while loop of method C
    data = await C()
    # Modify and process the data and return to A
    return await D(data)

async def C():
    i = 0
    while i < 5:
        await asyncio.sleep(1)
        # Return this data to method B one by one, Not sure how to do this ??
        return i

async def D(val):
    # Do some processing of val and return it
    return val

我希望从方法C连续传输数据并将其返回到方法B,处理收到的每个项目并将其返回到方法A.

一种方法是使用asyncio queue并将其从A传递给方法B,然后将其传递给C。

  1. 方法C会继续将内容写入队列。
  2. 方法B将从队列中读取,处理数据并更新队列。
  3. 方法A最后为最终处理的数据读取队列。
  4. 我们可以使用协同程序或异步方法本身以任何其他方式实现它吗?希望避免为每个请求连续读取和写入队列的呼叫。

1 个答案:

答案 0 :(得分:2)

import asyncio
from async_generator import async_generator, yield_, yield_from_

async def fun(n):
    print("Finding %d-1" % n)
    await asyncio.sleep(n/2)
    result = n - 1
    print("%d - 1 = %d" % (n, result))
    return result


@async_generator
async def main(l):
    futures = [ fun(n) for n in l ]
    for i, future in enumerate(asyncio.as_completed(futures)):
        result = await future
        print("inside the main..")
        print(result)
        await yield_(result)

@async_generator
async def dealer():
    l = [2, 4, 6]
    gen = main(l)
    async for item in gen:
        print("inside the dealer....")
        await yield_(item)

async def dealer1():
    gen = dealer()
    async for item in gen:
        print("inside dealer 1")
        print(item)

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    #loop.run_until_complete(cc.main())
    loop.run_until_complete(dealer1())
    loop.close()

您在python3.6中支持异步生成器。如果您使用的是python 3.5,则可以使用async_generator库(https://pypi.python.org/pypi/async_generator/1.5