KeyboardInterrupts与python的多处理池imap

时间:2014-09-11 09:20:04

标签: python multiprocessing imap keyboardinterrupt

按照此问题Keyboard Interrupts with python's multiprocessing Pool中的一些建议,编写名为test.py的测试代码

from multiprocessing import Pool
from multiprocessing.pool import IMapIterator
import fileinput

def wrapper(func):
  def wrap(self, timeout=None):
  # Note: the timeout of 1 googol seconds introduces a rather subtle
  # bug for Python scripts intended to run many times the age of the universe.
  return func(self, timeout=timeout if timeout is not None else 1e100)
return wrap
IMapIterator.next = wrapper(IMapIterator.next)

def main():
    p = Pool(1)
    try:
        for _ in p.imap(worker, fileinput.input(), 1):
            pass
    except KeyboardInterrupt:
        p.terminate()

def worker(line):
    print line

if __name__ == '__main__':
    main()

当我在终端中运行python test.py时:

python test.py

我仍然有KeyboardInterrupt回溯,而且我必须发送Ctrl ^ C两次才能停止脚本。我如何处理KeyboardInterrupt并让它不打印tarceback并只使用一次Ctrl ^ c来优雅地停止它。

^CProcess PoolWorker-1:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
KeyboardInterrupt
^CTraceback (most recent call last):
File "test.py", line 27, in <module>
    main()
File "test.py", line 21, in main
    p.terminate()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 452, in terminate
    self._worker_handler._state = TERMINATE
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/util.py", line 201, in __call__
    res = self._callback(*self._args, **self._kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 504, in _terminate_pool
    if threading.current_thread() is not task_handler:
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 958, in join
    self.__block.wait(delay)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 358, in wait
    _sleep(delay)
KeyboardInterrupt

1 个答案:

答案 0 :(得分:3)

查看标准库中包含的信号模块。您可以在主进程中注册信号处理程序

from signal import *

def siginthndlr(sig, frame):
    '''do what you need here'''
    print "Keyboard interrupt catched"

signal(SIGINT, siginthndlr) #Register SIGINT handler function

这将优雅地杀死工作进程而不是杀死主进程。