mpi4py表现得很奇怪

时间:2016-08-02 16:31:09

标签: python mpi pickle

尝试使用mpi4py在许多进程中加载​​pickle。这是我在大多数时间里工作得很好的东西:

for ix in range(len(frames)):
    if comm.rank == 0:
        frame = frames[ix]
        logging.debug("{0}: doing stuff, frame {1}".format(comm.rank,frame))
        W, nodes = pickle.load(open(os.path.join(args.network_dir, '{0}.pkl'.format(frame))))
        freqs = np.random.normal(0., 5.*np.pi/180., size = len(W))
    else:
        W = None
        nodes = None
        freqs = None

    comm.Barrier()

    W = comm.bcast(W, root = 0)
    nodes = comm.bcast(nodes, root = 0)
    freqs = comm.bcast(freqs, root = 0)

    comm.Barrier()

但它可靠地停留在广播步骤上。 freqs对于所有进程都必须相同,因此它必须由一个进程生成。我可以让每个人自己加载泡菜,如:

for ix in range(len(frames)):
    frame = frames[ix]
    logging.debug("{0}: doing stuff, frame {1}".format(comm.rank,frame))
    W, nodes = pickle.load(open(os.path.join(args.network_dir, '{0}.pkl'.format(frame))))

    if comm.rank == 0:
        freqs = np.random.normal(0., 5.*np.pi/180., size = len(W))
    else:
        freqs = None

    # broadcast and stuff

然后每个人都加载一个不同的泡菜!这不应该发生,对吧?每个进程都应该在ix = 0处进入循环,正确吗?

感谢任何帮助。

0 个答案:

没有答案