如何在python中使用多处理在进程内创建进程?

时间:2017-04-13 09:41:50

标签: python parallel-processing multiprocessing

我有一个需要2分钟才能运行的功能 我在我的程序中调用它两次,所以我使用多处理中的map来让它们并行运行。

现在我想减少更多的时间。

在这个功能中,我有8个可以并行运行的操作。

那么,我可以做的是让两个主要进程并行运行并且每个进程内部并行运行另外8个子进程。

1 个答案:

答案 0 :(得分:2)

我不知道为什么你需要这样的设置,但无论如何:

import _thread
import time

def operation(a, b, s):
    print("Starting operation {} in process {}".format(a,b))
    time.sleep(s)
    print("Finished operation {} in process {}".format(a,b))

def process(n):
    _thread.start_new_thread(operation, (1, n, 1))
    _thread.start_new_thread(operation, (2, n, 2))
    _thread.start_new_thread(operation, (3, n, 1))
    _thread.start_new_thread(operation, (4, n, 2))

_thread.start_new_thread(process, (1,))
_thread.start_new_thread(process, (2,))
time.sleep(3)

适用于Python 3.4。

编辑:正如所建议的那样,多处理可能更快,低于多处理的相同示例

from multiprocessing.pool import Pool as PoolParent
from multiprocessing import Process, Pool
import time

class NoDaemonProcess(Process):
    def _get_daemon(self):
        return False
    def _set_daemon(self, value):
        pass
    daemon = property(_get_daemon, _set_daemon)

class MyPool(PoolParent):
    Process = NoDaemonProcess

def operation(a):
    print("Starting operation {} in process {}".format(a[0],a[1]))
    time.sleep(a[2])
    print("Finished operation {} in process {}".format(a[0],a[1]))

def process(n):
    p = Pool(4)
    p.map(operation, [(1,n,1), (2,n,2), (3,n,1), (4,n,2)])

p = MyPool(2)
p.map(process, [1,2])
time.sleep(3)