Question

我有一个numpy数组，包含5000 28 x 28图像（5000,28,28）。我想将所有这些图像另存为jpg文件，并将它们全部保存在一个文件夹中。什么是最快，最有效的方法来完成此任务？

Answer 1

我尝试使用以下方式将50,000张28x28图像以JPEG格式写入磁盘：

顺序代码（25秒）
多线程代码（19秒）
多处理代码（5秒）

在具有SSD的12核MacBook Pro上。时间在上面列表中每个项目之后的括号中给出。

#!/usr/bin/env python3

import numpy as np
import imageio
from multiprocessing import Pool
from multiprocessing.pool import ThreadPool

def WriteOne(i,data):
    """Write a single image to disk"""
    #print(f"WriteOne called with i={i}")
    imageio.imwrite(f"image-{i:04d}.jpg", data)

def WriteSequential(images):
    """Write all images sequentially to disk"""
    print("Sequential")
    for i, data in enumerate(images):
        WriteOne(i, data)
    return

def WriteIndices(p):
    """Write given index to disk as JPEG"""
    #print(f"WriteIndices {p[0]}")
    WriteOne(p[0], p[1])
    return

def WriteMultiThread(images):
    """Write all images  to disk with multi-threading"""
    print("MultiThread")
    nThreads = 8
    with ThreadPool(nThreads) as pool:
        pool.map(WriteIndices, list(enumerate(images)))
    return

def WriteMultiProcess(images):
    """Write all images  to disk with multi-processing"""
    print("MultiProcess")
    nProcesses = 8
    with Pool(nProcesses) as pool:
        pool.map(WriteIndices, list(enumerate(images)))
    return

if __name__ == '__main__':

    # Synthesize 50000 images, each 28x28
    N = 50000
    images = np.random.randint(0,256,(N,28,28), dtype=np.uint8)

    WriteSequential(images)

    WriteMultiThread(images)

    WriteMultiProcess(images)

如何保存numpy数组图像并将其放入单个文件夹？

1 个答案: