Question

我有4个mpi进程，每个进程将写入多个连续的numpy数组。 data_per_rank 是一个二维numpy数组，每个mpi进程都应将其拆分为 inner_splits numpy子数组，并将每个数组写入不同的hdf5组。

inner_splits = 3
data_splits = np.array_split(data_per_rank, inner_splits, axis=0)

for i in range(size):
    for j in range(inner_splits):
        dset.append(f.create_dataset('p{0}_{1}'.format(i, j), data_splits[j].shape, dtype='float64'))

for i in range(inner_splits):
   dset[(rank*inner_splits)+i][:,:] = data_splits[i]

for i in range(size):
    del dset[i]

f.close()

这会产生以下错误：

File "/gpfs/projects/bsc19/bsc19004/matmul_separated.py", line 62, in write f.close()
File "/apps/PYTHON/2.7.13/INTEL/lib/python2.7/site-packages/h5py-2.7.0-py2.7-linux-x86_64.egg/h5py/_hl/files.py", line 299, in close
   h5i.dec_ref(id_)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/gpfs/apps/MN4/PYTHON/SRC/modules/2.7.13/h5py-.7.0_2/h5py/_objects.c:318)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/gpfs/apps/MN4/PYTHON/SRC/modules/2.7.13/h5py-.7.0_2/h5py/_objects.c:314)
File "h5py/h5i.pyx", line 150, in h5py.h5i.dec_ref(gpfs/apps/MN4/PYTHON/SRC/modules/2.7.13/h5py-2.7.0_2/h5py/h5i.c:2836)
RuntimeError: Can't decrement id ref count (Invalid argument, error stack:
mpi_file_set_size(82): inconsistent arguments to collective routine )

某些子数组的形状为（2731，32768），而其他子数组的形状为（2730，32768）。我确保在编写过程中将正确的data_split写入相应的dset处理程序。

有人可以向我解释这里出了什么问题吗？此错误消息是什么意思？哪些论据不一致？

预先感谢

h5py集体例程的参数不一致

0 个答案: