Question

我正在尝试在子过程中捕获异常。如果工作者数量==任务数量，我的代码（见下文）可以正常工作。但是，如果像示例中那样，工作人员多于任务，那么尚未启动的2个流程将抛出相同的异常，但是我既无法在MainProcess中也无法在子流程中捕获到它。

from multiprocessing import Pool, current_process
import time
from exceptions import GracefulExit
import signal


def terminate_handler(signum, frame):
    print(dir(frame))
    if current_process().name == 'MainProcess':
        # we will not raise error if process
        # is main because we need to finish all the jobs
        return
    raise GracefulExit()


def test_func(val):
    i = 0
    try:
        while True:
            time.sleep(0.1)
            i += 1
            if i >= 10:
                return i
    except GracefulExit:
        pass
    return i


if __name__ == "__main__":
    signal.signal(
            signal.SIGINT, terminate_handler)
    try:
        with Pool(5) as p:
            r = p.map(test_func, [1, 2, 3])
    except GracefulExit:
        pass
    print(r)

Answer 1

我可以建议什么，选择任何变体：

确保任务数不少于工作人员数，只需添加存根（例如None）以确保所有工作人员都可以开始

覆盖Pool过程的run方法：

class CustomProcess(Process):
    def run(self):
        try:
            super().run()
        except GracefulExit:
            print('interrupted idle worker')


class CustomPool(multiprocessing.pool.Pool):
    Process = CustomProcess

....

with CustomPool(5) as p:
   r = p.map(test_func, [1, 2, 3])

编写自己的池实现

在尚未启动的池工作程序中捕获异常

1 个答案: