Question

Python3.6现在asynchronous iterables。是否有内置方法将异步迭代转换为同步可迭代。

我目前有这个辅助功能，但感觉非常不像pythonic。有更好的方法吗？

async def aiter_to_list(aiter):
    l = []
    async for i in aiter:
        l.append(i)
    return l

Answer 1

你的＆＃34;异步到同步＆＃34;帮助器本身是异步的;根本不是一个很大的变化。一般来说：不，你不能让异步同步。将提供异步值＆＃34;稍后＆＃34 ;;你不能把它变成＆＃34;现在＆＃34;因为价值不存在＆＃34;现在＆＃34;并且你必须异步地等待它。

Answer 2

您可以使用aiostream.stream.list：

from aiostream import stream

async def agen():
    yield 1
    yield 2
    yield 3

async def main():
    lst = await stream.list(agen())
    print(lst)  # prints [1, 2, 3]

documentation中的更多运算符和示例。

Answer 3

这些功能允许您从/转换为iterable <==> async iterable，而不仅仅是简单的列表。

基本进口

import asyncio
import threading
import time

DONE = object()
TIMEOUT = 0.001

函数to_sync_iterable将所有异步可迭代转换为同步可迭代：

def to_sync_iterable(async_iterable, maxsize = 0):

    def sync_iterable():

        queue = asyncio.Queue(maxsize=maxsize)
        loop = asyncio.get_event_loop()

        t = threading.Thread(target=_run_coroutine, args=(loop, async_iterable, queue))
        t.daemon = True
        t.start()

        while True:
            if not queue.empty():
                x = queue.get_nowait()

                if x is DONE:
                    break
                else:
                    yield x
            else:
                time.sleep(utils.TIMEOUT)

        t.join()

    return sync_iterable()

def _run_coroutine(loop, async_iterable, queue):

    loop.run_until_complete(_consume_async_iterable(async_iterable, queue))

async def _consume_async_iterable(async_iterable, queue):

    async for x in async_iterable:
        await queue.put(x)

    await queue.put(DONE)

您可以像这样使用它：

async def slow_async_generator():
    yield 0

    await asyncio.sleep(1)
    yield 1

    await asyncio.sleep(1)
    yield 2

    await asyncio.sleep(1)
    yield 3


for x in to_sync_iterable(slow_async_generator()):
    print(x)

函数to_async_iterable会将所有可转换的同步迭代转换为异步可迭代：

def to_async_iterable(iterable, maxsize = 0):

    async def async_iterable():
        queue = asyncio.Queue(maxsize=maxsize)
        loop = asyncio.get_event_loop()
        task = loop.run_in_executor(None, lambda: _consume_iterable(loop, iterable, queue))

        while True:
            x = await queue.get()

            if x is DONE:
                break
            else:
                yield x

        await task


    return async_iterable()

def _consume_iterable(loop, iterable, queue):

    for x in iterable:
        while True:
            if not queue.full():
                loop.call_soon_threadsafe(queue.put_nowait, x)
                break
            else:
                time.sleep(TIMEOUT)

    while True:
        if not queue.full():
            loop.call_soon_threadsafe(queue.put_nowait, DONE)
            break
        else:
            time.sleep(TIMEOUT)

此选项对asyncio程序特别有用，因为即使sync可迭代，它也不会阻止事件循环。您可以像这样使用它：

def slow_sync_generator():
    yield 0

    time.sleep(1)
    yield 1

    time.sleep(1)
    yield 2

    time.sleep(1)
    yield 3

async def async_task():
    async for x in to_async_iterable(slow_sync_generator()):
        print(x)

asyncio.get_event_loop().run_until_complete(async_task())

Answer 4

我认为今天将会是：

async def async_list():
    for i in range(0,5):
        yield i

sync_list = [gen async for gen in async_list()]
print(sync_list) # [0, 1, 2, 3, 4]

构建将异步迭代转换为同步可迭代列表的方法

4 个答案: