如何在多进程python程序中捕获Ctrl + C并优雅地退出所有进程,我需要解决方案在unix和windows上工作。我尝试了以下内容:
import multiprocessing
import time
import signal
import sys
jobs = []
def worker():
signal.signal(signal.SIGINT, signal_handler)
while(True):
time.sleep(1.1234)
print "Working..."
def signal_handler(signal, frame):
print 'You pressed Ctrl+C!'
# for p in jobs:
# p.terminate()
sys.exit(0)
if __name__ == "__main__":
for i in range(50):
p = multiprocessing.Process(target=worker)
jobs.append(p)
p.start()
它有点工作,但我不认为这是正确的解决方案。
修改 这可能与this one
重复答案 0 :(得分:60)
The previously accepted solution具有竞争条件,但不适用于map
和async
个函数。
使用SIGINT
处理Ctrl + C / multiprocessing.Pool
的正确方法是:
SIGINT
之前,让流程忽略Pool
。这种方式创建了子进程继承SIGINT
处理程序。SIGINT
后,在父进程中还原原始Pool
处理程序。map_async
和apply_async
代替阻止map
和apply
。把它放在一起:
#!/bin/env python
from __future__ import print_function
import multiprocessing
import os
import signal
import time
def run_worker(delay):
print("In a worker process", os.getpid())
time.sleep(delay)
def main():
print("Initializng 2 workers")
original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN)
pool = multiprocessing.Pool(2)
signal.signal(signal.SIGINT, original_sigint_handler)
try:
print("Starting 2 jobs of 5 seconds each")
res = pool.map_async(run_worker, [5, 5])
print("Waiting for results")
res.get(60) # Without the timeout this blocking call ignores all signals.
except KeyboardInterrupt:
print("Caught KeyboardInterrupt, terminating workers")
pool.terminate()
else:
print("Normal termination")
pool.close()
pool.join()
if __name__ == "__main__":
main()
正如@YakovShklarov所指出的,在忽略信号和在父进程中取消它之间存在一段时间窗口,在此期间信号可能会丢失。使用pthread_sigmask
来暂时阻止父进程中的信号传递可以防止信号丢失,但是,它在Python-2中不可用。
答案 1 :(得分:34)
解决方案基于this link和this link,它解决了问题,但我不得不转移到Pool
:
import multiprocessing
import time
import signal
import sys
def init_worker():
signal.signal(signal.SIGINT, signal.SIG_IGN)
def worker():
while(True):
time.sleep(1.1234)
print "Working..."
if __name__ == "__main__":
pool = multiprocessing.Pool(50, init_worker)
try:
for i in range(50):
pool.apply_async(worker)
time.sleep(10)
pool.close()
pool.join()
except KeyboardInterrupt:
print "Caught KeyboardInterrupt, terminating workers"
pool.terminate()
pool.join()
答案 2 :(得分:12)
只需处理工作进程中的KeyboardInterrupt-SystemExit异常:
def worker():
while(True):
try:
msg = self.msg_queue.get()
except (KeyboardInterrupt, SystemExit):
print("Exiting...")
break