我想在子进程中使用Lock
来处理文件。当我在初始化程序中访问Lock
时,Pool
对初始化程序进行无限递归调用。
from multiprocessing import Lock, get_context
class MyClass(object):
@classmethod
def run_mp(cls, jobs):
context = get_context('spawn')
w_lock = Lock()
with context.Pool(1, initializer=cls._mp_init, initargs=(w_lock, )) as pool:
res = pool.map(cls._execute_job, jobs)
pool.close()
@classmethod
def _mp_init(cls, w_lock):
# global weights_lock
# weights_lock = w_lock
print('weights lock')
print(w_lock) # this line causes infinite calls to this method
@classmethod
def _execute_job(cls, job):
pass
if __name__ == '__main__':
jobs = [1]
mc = MyClass()
mc.run_mp(jobs)
预期产量
我在本地计算机上获得了预期的输出-mac_os,python3.7
weights lock
<Lock(owner=unknown)>
错误的输出
当我在gcloud(Linux 18.04,python 3.6)的linux实例上运行此命令时,我得到了
weights lock
weights lock
weights lock
weights lock
weights lock
# ... infinitely
如果我在print(w_lock)
方法中将_init_mp
注释掉,那么输出将再次如预期那样
weight lock
因此,基本上,当我尝试访问w_lock
时,它会进入_init_mp
的无限递归调用中。
我尝试在具有相同行为的不同glcoud实例上运行此程序。
我不明白为什么在Linux上访问锁会导致这种行为。有什么解释吗?