Question

我正在尝试将多个远程图像读入python并将这些图像读取为numpyarray，我尝试考虑使用异步方式来增强我的工作流程，但出现这样的错误：类型错误：numpy.ndarray对象不能在'await'表达式中使用，我想知道是因为方法readasarray不是异步的，所以如果我必须使其异步，则必须自己重写该方法。以下是我的一些代码：

async def taskIO_1():

    in_ds = gdal.Open(a[0])
    data1 = await in_ds.GetRasterBand(1).ReadAsArray()

async def taskIO_2():

    in_ds = gdal.Open(a[1])
    data2 = await in_ds.GetRasterBand(1).ReadAsArray()

async def main():

    tasks = [taskIO_1(), taskIO_2()]
    done,pending = await asyncio.wait(tasks)
    for r in done:
        print(r.result())

if __name__ == '__main__':
    start = time.time()
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(main())
    finally:
        loop.close()
    print(float(time.time()-start))

Answer 1

您的概念是正确的：通常，除非以明确的方式编写库以支持异步执行（例如，通过使用非阻塞I / O），否则库函数以同步（阻塞）方式执行，例如{{3 }}或aiofiles。

要使用要异步执行的同步调用，可以使用loop.run_in_executor。除了将计算分流到单独的线程或进程中并将其包装起来，使其表现得像协程一样，它无济于事。 aiohttp显示了一个示例：

import asyncio
import concurrent.futures

def blocking_io():
    # File operations (such as logging) can block the
    # event loop: run them in a thread pool.
    with open('/dev/urandom', 'rb') as f:
        return f.read(100)

def cpu_bound():
    # CPU-bound operations will block the event loop:
    # in general it is preferable to run them in a
    # process pool.
    return sum(i * i for i in range(10 ** 7))

async def main():
    loop = asyncio.get_running_loop()

    ## Options:

    # 1. Run in the default loop's executor:
    result = await loop.run_in_executor(
        None, blocking_io)
    print('default thread pool', result)

    # 2. Run in a custom thread pool:
    with concurrent.futures.ThreadPoolExecutor() as pool:
        result = await loop.run_in_executor(
            pool, blocking_io)
        print('custom thread pool', result)

    # 3. Run in a custom process pool:
    with concurrent.futures.ProcessPoolExecutor() as pool:
        result = await loop.run_in_executor(
            pool, cpu_bound)
        print('custom process pool', result)

asyncio.run(main())

但是，如果您的应用程序未使用任何真正的异步功能，则最好直接使用concurrent.futures池并以这种方式实现并发。

为什么我不能等待gdal模块中的readasarray方法？

1 个答案: