Question

我有一个大的3d numpy数组，每个切片（2d数组）我想写出一个类似于imshow的数字（即值的热图）。作为一个具体的例子，假设阵列的形状是3x3x3000，所以我想要3000个图像，每个图像代表一个3x3矩阵。用单个线程循环它有点慢。由于迭代是完全独立的，我想使用多处理模块来加速它。代码如下。

def write_tensor_image(t_slice_wrapper):

    idx = t_slice_wrapper['idx']
    t_slice = t_slice_wrapper['t_slice']
    folder_path=t_slice_wrapper['folder_path']

    fig = matplotlib.pyplot.figure()
    ax = fig.add_subplot(111)    
    ax.imshow(t_slice,interpolation='none')
    fig.tight_layout()

    fname_ = os.path.join(folder_path,'tmp_%s.png'%str(idx))
    fig.savefig(fname_, bbox_inches="tight")    

def write_tensor_image_sequence(tensor, folder_path='/home/foo/numpy_cache'):

    os.system('mkdir -p %s'%folder_path)
    os.system('rm -rf %s/*'%folder_path)

    slices = [None]*tensor.shape[2]
    for i in range(0,tensor.shape[2]):
        slices[i] = {'t_slice':tensor[:,:,i], 'idx':i, 'folder_path':folder_path}

    pool = multiprocessing.Pool(processes=4)
    pool.map(write_tensor_image, slices)
    pool.close()
    pool.join()

但是这不起作用 - 单线程情况工作正常（只是在for循环中调用write_tensor_image（））但是使用该池会导致机器完全锁定或者出现类似以下错误：

XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
  after 849 requests (849 known processed) with 28 events remaining.
XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
  after 849 requests (849 known processed) with 28 events remaining.
  after 849 requests (849 known processed) with 28 events remaining.
X Error of failed request:  BadPixmap (invalid Pixmap parameter)
Major opcode of failed request:  54 (X_FreePixmap)
Resource id in failed request:  0x4e0001e
Serial number of failed request:  851
Current serial number in output stream:  851

我认为我走在正确的轨道上（例如How to fix the python multiprocessing matplotlib savefig() issue?和Matplotlib: simultaneous plotting in multiple threads），但我必须遗漏一些东西。

Answer 1

在脚本的头部做一个matplotlib.use('agg')。 Matplotlib似乎试图在每个子流程中建立一个GUI，它们彼此冲突。

更一般地说，在您没有进行标准交互式绘图的情况下，您可能不想使用pyplot界面，而是使用OOP界面。

Python多处理+ savefig导致错误或系统锁定

1 个答案: