捕获Ctrl + C / SIGINT并在python中正常退出多进程

时间:2012-07-03 13:56:49

标签: python multiprocessing signals

如何在多进程python程序中捕获Ctrl + C并优雅地退出所有进程,我需要解决方案在unix和windows上工作。我尝试了以下内容:

import multiprocessing
import time
import signal
import sys

jobs = []

def worker():
    signal.signal(signal.SIGINT, signal_handler)
    while(True):
        time.sleep(1.1234)
        print "Working..."

def signal_handler(signal, frame):
    print 'You pressed Ctrl+C!'
    # for p in jobs:
    #     p.terminate()
    sys.exit(0)

if __name__ == "__main__":
    for i in range(50):
        p = multiprocessing.Process(target=worker)
        jobs.append(p)
        p.start()

它有点工作,但我不认为这是正确的解决方案。

修改 这可能与this one

重复

3 个答案:

答案 0 :(得分:60)

The previously accepted solution具有竞争条件,但不适用于mapasync个函数。

使用SIGINT处理Ctrl + C / multiprocessing.Pool的正确方法是:

  1. 在创建流程SIGINT之前,让流程忽略Pool。这种方式创建了子进程继承SIGINT处理程序。
  2. 在创建SIGINT后,在父进程中还原原始Pool处理程序。
  3. 使用map_asyncapply_async代替阻止mapapply
  4. 等待结果超时,因为默认阻止等待忽略所有信号。这是Python错误https://bugs.python.org/issue8296
  5. 把它放在一起:

    #!/bin/env python
    from __future__ import print_function
    
    import multiprocessing
    import os
    import signal
    import time
    
    def run_worker(delay):
        print("In a worker process", os.getpid())
        time.sleep(delay)
    
    def main():
        print("Initializng 2 workers")
        original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN)
        pool = multiprocessing.Pool(2)
        signal.signal(signal.SIGINT, original_sigint_handler)
        try:
            print("Starting 2 jobs of 5 seconds each")
            res = pool.map_async(run_worker, [5, 5])
            print("Waiting for results")
            res.get(60) # Without the timeout this blocking call ignores all signals.
        except KeyboardInterrupt:
            print("Caught KeyboardInterrupt, terminating workers")
            pool.terminate()
        else:
            print("Normal termination")
            pool.close()
        pool.join()
    
    if __name__ == "__main__":
        main()
    

    正如@YakovShklarov所指出的,在忽略信号和在父进程中取消它之间存在一段时间窗口,在此期间信号可能会丢失。使用pthread_sigmask来暂时阻止父进程中的信号传递可以防止信号丢失,但是,它在Python-2中不可用。

答案 1 :(得分:34)

解决方案基于this linkthis link,它解决了问题,但我不得不转移到Pool

import multiprocessing
import time
import signal
import sys

def init_worker():
    signal.signal(signal.SIGINT, signal.SIG_IGN)

def worker():
    while(True):
        time.sleep(1.1234)
        print "Working..."

if __name__ == "__main__":
    pool = multiprocessing.Pool(50, init_worker)
    try:
        for i in range(50):
            pool.apply_async(worker)

        time.sleep(10)
        pool.close()
        pool.join()

    except KeyboardInterrupt:
        print "Caught KeyboardInterrupt, terminating workers"
        pool.terminate()
        pool.join()

答案 2 :(得分:12)

只需处理工作进程中的KeyboardInterrupt-SystemExit异常:

def worker():
    while(True):
        try:
            msg = self.msg_queue.get()
        except (KeyboardInterrupt, SystemExit):
            print("Exiting...")
            break