Question

我有一个非常简单的MPI脚本使用mpi4py

# mpitest.py
from mpi4py import MPI
import time

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

time.sleep(100)

如果我使用mpirun正常运行，一切正常

$ mpirun --np 4 python mpitest.py  # just fine

但是如果我使用子进程模块在Python中运行它，那么事情就会运行，但是我的解释器变得非常迟缓

>>> import subprocess
>>> proc = subprocess.Popen(['mpirun', '--np', '2', 'python', 'mpitest.py'])

我已经尝试过像shell=True这样的关键字参数。

环境

我使用最新的Miniconda for Linux安装了Python，mpi4py和mpich

mrocklin@carbon:~/workspace/play$ conda list | grep mpi
mpi4py                    2.0.0                    py36_2  
mpich2                    1.4.1p1                       0

https://conda.io/miniconda.html

可重复的步骤

mrocklin@carbon:~/workspace/play$ conda create -n test-mpi python=3.6 mpi4py
Fetching package metadata .........
Solving package specifications: .

Package plan for installation in environment /home/mrocklin/Software/anaconda/envs/test-mpi:

The following NEW packages will be INSTALLED:

    mpi4py:     2.0.0-py36_2 
    mpich2:     1.4.1p1-0    
    openssl:    1.0.2l-0     
    pip:        9.0.1-py36_1 
    python:     3.6.2-0      
    readline:   6.2-2        
    setuptools: 27.2.0-py36_0
    sqlite:     3.13.0-0     
    tk:         8.5.18-0     
    wheel:      0.29.0-py36_0
    xz:         5.2.3-0      
    zlib:       1.2.11-0     

xz-5.2.3-0.tar 100% |################################| Time: 0:00:00   3.79 MB/s
zlib-1.2.11-0. 100% |################################| Time: 0:00:00   5.68 MB/s
#
# To activate this environment, use:
# > source activate test-mpi
#
# To deactivate an active environment, use:
# > source deactivate
#

mrocklin@carbon:~/workspace/play$ source activate test-mpi
(test-mpi) mrocklin@carbon:~/workspace/play$ python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> proc = subprocess.Popen(['mpirun', '--np', '2', 'python', 'mpitest.py'])

Answer 1

可以通过将stdin=subprocess.DEVNULL关键字添加到subprocess.Popen调用来解决此问题，如下所示：

>>> proc = subprocess.Popen(['mpirun', '--np', '2', 'python', 'mpitest.py'], 
                            stdin=subprocess.DEVNULL)

事实证明mpirun劫持了stdin管道，然后使得python进程的许多击键都没有到达。

为什么Python在子进程中运行mpirun时会挂起？

环境

可重复的步骤

1 个答案: