我在Python中调用一个函数，我知道它可能会停止并迫使我重新启动脚本。
如何调用该函数或将其包装成什么内容，以便在脚本取消时间超过5秒后再执行其他操作？
答案 0 :(得分：186)
如果您在UNIX上运行，则可以使用signal包：
In [1]: import signal
# Register an handler for the timeout
In [2]: def handler(signum, frame):
...: print "Forever is over!"
...: raise Exception("end of time")
...:
# This function *may* run for an indetermined time...
In [3]: def loop_forever():
...: import time
...: while 1:
...: print "sec"
...: time.sleep(1)
...:
...:
# Register the signal function handler
In [4]: signal.signal(signal.SIGALRM, handler)
Out[4]: 0
# Define a timeout for your function
In [5]: signal.alarm(10)
Out[5]: 0
In [6]: try:
...: loop_forever()
...: except Exception, exc:
...: print exc
....:
sec
sec
sec
sec
sec
sec
sec
sec
Forever is over!
end of time
# Cancel the timer if the function returned before timeout
# (ok, mine won't but yours maybe will :)
In [7]: signal.alarm(0)
Out[7]: 0
调用alarm.alarm(10)后10秒，调用处理程序。这引发了一个异常，你可以从常规的Python代码中拦截它。
这个模块不能很好地与线程一起使用（但是，谁呢？）
注意，因为我们在超时发生时引发异常，它可能最终被捕获并在函数内被忽略，例如一个这样的函数：
def loop_forever():
while 1:
print 'sec'
try:
time.sleep(10)
except:
continue
答案 1 :(得分：128)
您可以使用multiprocessing.Process来做到这一点。
<强>代码
import multiprocessing
import time
# bar
def bar():
for i in range(100):
print "Tick"
time.sleep(1)
if __name__ == '__main__':
# Start bar as a process
p = multiprocessing.Process(target=bar)
p.start()
# Wait for 10 seconds or until process finishes
p.join(10)
# If thread is still active
if p.is_alive():
print "running... let's kill it..."
# Terminate
p.terminate()
p.join()
答案 2 :(得分：59)
如何调用该函数或将其包装成什么内容，以便在脚本取消时间超过5秒时将其取消？
我发布了gist，用装饰器和threading.Timer解决了这个问题/问题。这是故障。
它使用Python 2和3进行了测试。它也应该在Unix / Linux和Windows下运行。
首先是进口。无论Python版本如何，这些都试图保持代码一致：
from __future__ import print_function
import sys
import threading
from time import sleep
try:
import thread
except ImportError:
import _thread as thread
使用与版本无关的代码：
try:
range, _print = xrange, print
def print(*args, **kwargs):
flush = kwargs.pop('flush', False)
_print(*args, **kwargs)
if flush:
kwargs.get('file', sys.stdout).flush()
except NameError:
pass
现在我们已经从标准库导入了我们的功能。
exit_after装饰者接下来，我们需要一个函数来终止子线程中的main()：
def quit_function(fn_name):
# print to stderr, unbuffered in Python 2.
print('{0} took too long'.format(fn_name), file=sys.stderr)
sys.stderr.flush() # Python 3 stderr is likely buffered.
thread.interrupt_main() # raises KeyboardInterrupt
以下是装饰者本身：
def exit_after(s):
'''
use as decorator to exit process if
function takes longer than s seconds
'''
def outer(fn):
def inner(*args, **kwargs):
timer = threading.Timer(s, quit_function, args=[fn.__name__])
timer.start()
try:
result = fn(*args, **kwargs)
finally:
timer.cancel()
return result
return inner
return outer
这里的用法直接回答了你在5秒后退出的问题！：
@exit_after(5)
def countdown(n):
print('countdown started', flush=True)
for i in range(n, -1, -1):
print(i, end=', ', flush=True)
sleep(1)
print('countdown finished')
演示：
>>> countdown(3)
countdown started
3, 2, 1, 0, countdown finished
>>> countdown(10)
countdown started
10, 9, 8, 7, 6, countdown took too long
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in inner
File "<stdin>", line 6, in countdown
KeyboardInterrupt
第二个函数调用不会完成，而是进程应该以traceback退出！
KeyboardInterrupt并不总是停止睡眠线程请注意，在Windows上的Python 2上，键盘中断并不总是会中断睡眠，例如：
@exit_after(1)
def sleep10():
sleep(10)
print('slept 10 seconds')
>>> sleep10()
sleep10 took too long # Note that it hangs here about 9 more seconds
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in inner
File "<stdin>", line 3, in sleep10
KeyboardInterrupt
也不可能中断在扩展程序中运行的代码，除非它明确检查PyErr_CheckSignals()，请参阅Cython, Python and KeyboardInterrupt ignored
在任何情况下，我都会避免在一个线程内休息一秒钟 - 处理器时间已经过去了。
如何调用该函数或将其包装在哪里，以便在脚本取消时间超过5秒时并执行其他操作？
要捕获它并执行其他操作，您可以捕获KeyboardInterrupt。
>>> try:
... countdown(10)
... except KeyboardInterrupt:
... print('do something else')
...
countdown started
10, 9, 8, 7, 6, countdown took too long
do something else
答案 3 :(得分：44)
我有一个不同的提议，它是一个纯函数（使用与线程建议相同的API）并且似乎工作正常（基于此线程的建议）
def timeout(func, args=(), kwargs={}, timeout_duration=1, default=None):
import signal
class TimeoutError(Exception):
pass
def handler(signum, frame):
raise TimeoutError()
# set the timeout handler
signal.signal(signal.SIGALRM, handler)
signal.alarm(timeout_duration)
try:
result = func(*args, **kwargs)
except TimeoutError as exc:
result = default
finally:
signal.alarm(0)
return result
答案 4 :(得分：29)
我在单元测试中搜索超时调用时遇到了这个线程。我在答案或第三方软件包中找不到任何简单的内容，所以我在下面写了装饰器，你可以直接进入代码：
import multiprocessing.pool
import functools
def timeout(max_timeout):
"""Timeout decorator, parameter in seconds."""
def timeout_decorator(item):
"""Wrap the original function."""
@functools.wraps(item)
def func_wrapper(*args, **kwargs):
"""Closure for function."""
pool = multiprocessing.pool.ThreadPool(processes=1)
async_result = pool.apply_async(item, args, kwargs)
# raises a TimeoutError if execution exceeds max_timeout
return async_result.get(max_timeout)
return func_wrapper
return timeout_decorator
然后就这么简单来超时测试或任何你喜欢的功能：
@timeout(5.0) # if execution takes longer than 5 seconds, raise a TimeoutError
def test_base_regression(self):
...
答案 5 :(得分：14)
有很多建议，但没有使用concurrent.futures，我认为这是最清晰的处理方法。
from concurrent.futures import ProcessPoolExecutor
# Warning: this does not terminate function if timeout
def timeout_five(fnc, *args, **kwargs):
with ProcessPoolExecutor() as p:
f = p.submit(fnc, *args, **kwargs)
return f.result(timeout=5)
超级简单易读和维护。
我们创建一个池，提交一个进程，然后等待最多5秒钟，然后再提出一个TimeoutError，你可以捕获并处理你需要的。
原生于python 3.2+并向后移植到2.7（pip install期货）。
在线程和流程之间切换就像用ProcessPoolExecutor替换ThreadPoolExecutor一样简单。
如果您想在超时时终止进程，我建议您查看Pebble。
答案 6 :(得分：12)
在pypi上找到的stopit包似乎很好地处理了超时。
我喜欢@stopit.threading_timeoutable装饰器，它为装饰函数添加了一个timeout参数，它可以满足你的期望，它会停止这个功能。
在pypi上查看：https://pypi.python.org/pypi/stopit
答案 7 :(得分：5)
出色，易于使用且可靠的 PyPi 项目超时装饰器（https://pypi.org/project/timeout-decorator/）
安装：
pip install timeout-decorator
用法：
import time
import timeout_decorator
@timeout_decorator.timeout(5)
def mytest():
print "Start"
for i in range(1,10):
time.sleep(1)
print "%d seconds have passed" % i
if __name__ == '__main__':
mytest()
答案 8 :(得分：3)
通过@piro建立并增强答案，您可以构建一个contextmanager。这允许非常易读的代码，该代码将在成功运行后禁用alaram信号（设置signal.alarm（0））
@contextmanager
def timeout(duration):
def timeout_handler(signum, frame):
raise Exception(f'block timedout after {duration} seconds')
signal.signal(signal.SIGALRM, timeout_handler)
signal.alarm(duration)
yield
signal.alarm(0)
def sleeper(duration):
time.sleep(duration)
print('finished')
用法示例：
In [19]: with timeout(2):
...: sleeper(1)
...:
finished
In [20]: with timeout(2):
...: sleeper(3)
...:
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-20-66c78858116f> in <module>()
1 with timeout(2):
----> 2 sleeper(3)
3
<ipython-input-7-a75b966bf7ac> in sleeper(t)
1 def sleeper(t):
----> 2 time.sleep(t)
3 print('finished')
4
<ipython-input-18-533b9e684466> in timeout_handler(signum, frame)
2 def timeout(duration):
3 def timeout_handler(signum, frame):
----> 4 raise Exception(f'block timedout after {duration} seconds')
5 signal.signal(signal.SIGALRM, timeout_handler)
6 signal.alarm(duration)
Exception: block timedout after 2 seconds
答案 9 :(得分：3)
使用asyncio的另一种解决方案：
import asyncio
import functools
import multiprocessing
from concurrent.futures.thread import ThreadPoolExecutor
class SingletonTimeOut:
pool = None
@classmethod
def run(cls, to_run: functools.partial, timeout: float):
pool = cls.get_pool()
loop = cls.get_loop()
try:
task = loop.run_in_executor(pool, to_run)
return loop.run_until_complete(asyncio.wait_for(task, timeout=timeout))
except asyncio.TimeoutError as e:
error_type = type(e).__name__ #TODO
raise e
@classmethod
def get_pool(cls):
if cls.pool is None:
cls.pool = ThreadPoolExecutor(multiprocessing.cpu_count())
return cls.pool
@classmethod
def get_loop(cls):
try:
return asyncio.get_event_loop()
except RuntimeError:
asyncio.set_event_loop(asyncio.new_event_loop())
# print("NEW LOOP" + str(threading.current_thread().ident))
return asyncio.get_event_loop()
# ---------------
TIME_OUT = float('0.2') # seconds
def toto(input_items,nb_predictions):
return 1
to_run = functools.partial(toto,
input_items=1,
nb_predictions="a")
results = SingletonTimeOut.run(to_run, TIME_OUT)
答案 10 :(得分：2)
#!/usr/bin/python2
import sys, subprocess, threading
proc = subprocess.Popen(sys.argv[2:])
timer = threading.Timer(float(sys.argv[1]), proc.terminate)
timer.start()
proc.wait()
timer.cancel()
exit(proc.returncode)
答案 11 :(得分：2)
以防万一它对任何人有帮助，在@piro 的回答基础上，我做了一个函数装饰器：
import time
import signal
from functools import wraps
def timeout(timeout_secs: int):
def wrapper(func):
@wraps(func)
def time_limited(*args, **kwargs):
# Register an handler for the timeout
def handler(signum, frame):
raise Exception(f"Timeout for function '{func.__name__}'")
# Register the signal function handler
signal.signal(signal.SIGALRM, handler)
# Define a timeout for your function
signal.alarm(timeout_secs)
result = None
try:
result = func(*args, **kwargs)
except Exception as exc:
raise exc
finally:
# disable the signal alarm
signal.alarm(0)
return result
return time_limited
return wrapper
在具有 20 seconds 超时的函数上使用包装器看起来类似于：
@timeout(20)
def my_slow_or_never_ending_function(name):
while True:
time.sleep(1)
print(f"Yet another second passed {name}...")
try:
results = my_slow_or_never_ending_function("Yooo!")
except Exception as e:
print(f"ERROR: {e}")
答案 12 :(得分：1)
我需要可嵌套定时中断（SIGALARM无法做到），这些中断不会被time.sleep阻塞（基于线程的方法可以＆＃39） ; t）。我最终复制并轻轻修改了代码：http://code.activestate.com/recipes/577600-queue-for-managing-multiple-sigalrm-alarms-concurr/
代码本身：
#!/usr/bin/python
# lightly modified version of http://code.activestate.com/recipes/577600-queue-for-managing-multiple-sigalrm-alarms-concurr/
"""alarm.py: Permits multiple SIGALRM events to be queued.
Uses a `heapq` to store the objects to be called when an alarm signal is
raised, so that the next alarm is always at the top of the heap.
"""
import heapq
import signal
from time import time
__version__ = '$Revision: 2539 $'.split()[1]
alarmlist = []
__new_alarm = lambda t, f, a, k: (t + time(), f, a, k)
__next_alarm = lambda: int(round(alarmlist[0][0] - time())) if alarmlist else None
__set_alarm = lambda: signal.alarm(max(__next_alarm(), 1))
class TimeoutError(Exception):
def __init__(self, message, id_=None):
self.message = message
self.id_ = id_
class Timeout:
''' id_ allows for nested timeouts. '''
def __init__(self, id_=None, seconds=1, error_message='Timeout'):
self.seconds = seconds
self.error_message = error_message
self.id_ = id_
def handle_timeout(self):
raise TimeoutError(self.error_message, self.id_)
def __enter__(self):
self.this_alarm = alarm(self.seconds, self.handle_timeout)
def __exit__(self, type, value, traceback):
try:
cancel(self.this_alarm)
except ValueError:
pass
def __clear_alarm():
"""Clear an existing alarm.
If the alarm signal was set to a callable other than our own, queue the
previous alarm settings.
"""
oldsec = signal.alarm(0)
oldfunc = signal.signal(signal.SIGALRM, __alarm_handler)
if oldsec > 0 and oldfunc != __alarm_handler:
heapq.heappush(alarmlist, (__new_alarm(oldsec, oldfunc, [], {})))
def __alarm_handler(*zargs):
"""Handle an alarm by calling any due heap entries and resetting the alarm.
Note that multiple heap entries might get called, especially if calling an
entry takes a lot of time.
"""
try:
nextt = __next_alarm()
while nextt is not None and nextt <= 0:
(tm, func, args, keys) = heapq.heappop(alarmlist)
func(*args, **keys)
nextt = __next_alarm()
finally:
if alarmlist: __set_alarm()
def alarm(sec, func, *args, **keys):
"""Set an alarm.
When the alarm is raised in `sec` seconds, the handler will call `func`,
passing `args` and `keys`. Return the heap entry (which is just a big
tuple), so that it can be cancelled by calling `cancel()`.
"""
__clear_alarm()
try:
newalarm = __new_alarm(sec, func, args, keys)
heapq.heappush(alarmlist, newalarm)
return newalarm
finally:
__set_alarm()
def cancel(alarm):
"""Cancel an alarm by passing the heap entry returned by `alarm()`.
It is an error to try to cancel an alarm which has already occurred.
"""
__clear_alarm()
try:
alarmlist.remove(alarm)
heapq.heapify(alarmlist)
finally:
if alarmlist: __set_alarm()
和一个用法示例：
import alarm
from time import sleep
try:
with alarm.Timeout(id_='a', seconds=5):
try:
with alarm.Timeout(id_='b', seconds=2):
sleep(3)
except alarm.TimeoutError as e:
print 'raised', e.id_
sleep(30)
except alarm.TimeoutError as e:
print 'raised', e.id_
else:
print 'nope.'
答案 13 :(得分：1)
timeout-decorator在Windows系统上无法正常运行，因为Windows不能很好地支持signal。
如果在Windows系统中使用超时装饰器，则会显示以下内容
AttributeError: module 'signal' has no attribute 'SIGALRM'
有人建议使用use_signals=False，但对我没有用。
作者@bitranox创建了以下程序包：
pip install https://github.com/bitranox/wrapt-timeout-decorator/archive/master.zip
代码示例：
import time
from wrapt_timeout_decorator import *
@timeout(5)
def mytest(message):
print(message)
for i in range(1,10):
time.sleep(1)
print('{} seconds have passed'.format(i))
def main():
mytest('starting')
if __name__ == '__main__':
main()
给出以下异常：
TimeoutError: Function mytest timed out after 5 seconds
答案 14 :(得分：1)
我是wrapt_timeout_decorator的作者
乍一看，这里介绍的大多数解决方案在Linux上都无法正常工作-因为我们有fork（）和signal（）-但在Windows上看起来有些不同。 当涉及到Linux上的子线程时，您不能再使用Signals。
要在Windows下生成一个进程，它必须是可选取的-许多修饰的函数或Class方法不是。
因此，您需要使用更好的Picker，如莳萝和多进程（而不是pickle和多进程）-这就是为什么您不能使用ProcessPoolExecutor（或仅具有有限功能）的原因。
对于超时本身-您需要定义超时的含义-因为在Windows上将花费大量（且不确定）的时间来生成该进程。如果超时时间短，这可能会很棘手。假设产生该过程大约需要0.5秒（很容易！！！）。如果您给出0.2秒的超时时间，应该怎么办？ 函数是否应在0.5 + 0.2秒后超时（让方法运行0.2秒）？ 还是被调用的进程应该在0.2秒后超时（在这种情况下，修饰的函数将始终超时，因为在那个时间内它甚至没有生成）？
嵌套装饰器也很讨厌，您不能在子线程中使用Signals。如果要创建真正的通用跨平台装饰器，则需要考虑所有这些因素（并进行测试）。
其他问题会将异常传递回调用者，以及记录问题（如果在修饰的函数中使用-不支持在另一个进程中记录文件）
我试图涵盖所有的极端情况，您可能会研究wrapt_timeout_decorator包，或者至少测试一下在那里使用的单元测试启发的自己的解决方案。
@Alexis Eggermont-很遗憾，我没有足够的意见要发表-也许其他人可以通知您-我想我已经解决了您的导入问题。
答案 15 :(得分：1)
我们可以使用相同的信号。我认为以下示例对您有用。与线程相比，它非常简单。
import signal
def timeout(signum, frame):
raise myException
#this is an infinite loop, never ending under normal circumstances
def main():
print 'Starting Main ',
while 1:
print 'in main ',
#SIGALRM is only usable on a unix platform
signal.signal(signal.SIGALRM, timeout)
#change 5 to however many seconds you need
signal.alarm(5)
try:
main()
except myException:
print "whoops"
答案 16 :(得分：1)
TimeoutError 使用异常来提醒超时-可以轻松修改有关并行映射的完整说明和扩展，请参见https://flipdazed.github.io/blog/quant%20dev/parallel-functions-with-timeouts
>>> @killer_call(timeout=4)
... def bar(x):
... import time
... time.sleep(x)
... return x
>>> bar(10)
Traceback (most recent call last):
...
__main__.TimeoutError: function 'bar' timed out after 4s
并按预期
>>> bar(2)
2
import multiprocessing as mp
import multiprocessing.queues as mpq
import functools
import dill
from typing import Tuple, Callable, Dict, Optional, Iterable, List
class TimeoutError(Exception):
def __init__(self, func, timeout):
self.t = timeout
self.fname = func.__name__
def __str__(self):
return f"function '{self.fname}' timed out after {self.t}s"
def _lemmiwinks(func: Callable, args: Tuple[object], kwargs: Dict[str, object], q: mp.Queue):
"""lemmiwinks crawls into the unknown"""
q.put(dill.loads(func)(*args, **kwargs))
def killer_call(func: Callable = None, timeout: int = 10) -> Callable:
"""
Single function call with a timeout
Args:
func: the function
timeout: The timeout in seconds
"""
if not isinstance(timeout, int):
raise ValueError(f'timeout needs to be an int. Got: {timeout}')
if func is None:
return functools.partial(killer_call, timeout=timeout)
@functools.wraps(killer_call)
def _inners(*args, **kwargs) -> object:
q_worker = mp.Queue()
proc = mp.Process(target=_lemmiwinks, args=(dill.dumps(func), args, kwargs, q_worker))
proc.start()
try:
return q_worker.get(timeout=timeout)
except mpq.Empty:
raise TimeoutError(func, timeout)
finally:
try:
proc.terminate()
except:
pass
return _inners
if __name__ == '__main__':
@killer_call(timeout=4)
def bar(x):
import time
time.sleep(x)
return x
print(bar(2))
bar(10)
由于dill的工作方式，您需要在函数内部导入。
这也意味着，如果目标函数内部有导入，则这些函数可能与doctest不兼容。您会遇到找不到__import__的问题。
答案 17 :(得分：0)
我遇到了同样的问题，但我的情况是需要在子线程上工作，信号对我不起作用，所以我写了一个python包：timeout-timer来解决这个问题，支持用作上下文或装饰器，使用信号或子线程模块触发超时中断：
from timeout_timer import timeout, TimeoutInterrupt
class TimeoutInterruptNested(TimeoutInterrupt):
pass
def test_timeout_nested_loop_both_timeout(timer="thread"):
cnt = 0
try:
with timeout(5, timer=timer):
try:
with timeout(2, timer=timer, exception=TimeoutInterruptNested):
sleep(2)
except TimeoutInterruptNested:
cnt += 1
time.sleep(10)
except TimeoutInterrupt:
cnt += 1
assert cnt == 2
答案 18 :(得分：0)
这里是POSIX版本，结合了许多以前的答案，可以提供以下功能：
这是代码和一些测试用例：
import threading
import signal
import os
import time
class TerminateExecution(Exception):
"""
Exception to indicate that execution has exceeded the preset running time.
"""
def quit_function(pid):
# Killing all subprocesses
os.setpgrp()
os.killpg(0, signal.SIGTERM)
# Killing the main thread
os.kill(pid, signal.SIGTERM)
def handle_term(signum, frame):
raise TerminateExecution()
def invoke_with_timeout(timeout, fn, *args, **kwargs):
# Setting a sigterm handler and initiating a timer
old_handler = signal.signal(signal.SIGTERM, handle_term)
timer = threading.Timer(timeout, quit_function, args=[os.getpid()])
terminate = False
# Executing the function
timer.start()
try:
result = fn(*args, **kwargs)
except TerminateExecution:
terminate = True
finally:
# Restoring original handler and cancel timer
signal.signal(signal.SIGTERM, old_handler)
timer.cancel()
if terminate:
raise BaseException("xxx")
return result
### Test cases
def countdown(n):
print('countdown started', flush=True)
for i in range(n, -1, -1):
print(i, end=', ', flush=True)
time.sleep(1)
print('countdown finished')
return 1337
def really_long_function():
time.sleep(10)
def really_long_function2():
os.system("sleep 787")
# Checking that we can run a function as expected.
assert invoke_with_timeout(3, countdown, 1) == 1337
# Testing various scenarios
t1 = time.time()
try:
print(invoke_with_timeout(1, countdown, 3))
assert(False)
except BaseException:
assert(time.time() - t1 < 1.1)
print("All good", time.time() - t1)
t1 = time.time()
try:
print(invoke_with_timeout(1, really_long_function2))
assert(False)
except BaseException:
assert(time.time() - t1 < 1.1)
print("All good", time.time() - t1)
t1 = time.time()
try:
print(invoke_with_timeout(1, really_long_function))
assert(False)
except BaseException:
assert(time.time() - t1 < 1.1)
print("All good", time.time() - t1)
# Checking that classes are referenced and not
# copied (as would be the case with multiprocessing)
class X:
def __init__(self):
self.value = 0
def set(self, v):
self.value = v
x = X()
invoke_with_timeout(2, x.set, 9)
assert x.value == 9
答案 19 :(得分：0)
这是对给定基于线程的解决方案的一点改进。
以下代码支持例外：
def runFunctionCatchExceptions(func, *args, **kwargs):
try:
result = func(*args, **kwargs)
except Exception, message:
return ["exception", message]
return ["RESULT", result]
def runFunctionWithTimeout(func, args=(), kwargs={}, timeout_duration=10, default=None):
import threading
class InterruptableThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.result = default
def run(self):
self.result = runFunctionCatchExceptions(func, *args, **kwargs)
it = InterruptableThread()
it.start()
it.join(timeout_duration)
if it.isAlive():
return default
if it.result[0] == "exception":
raise it.result[1]
return it.result[1]
以5秒超时调用它：
result = timeout(remote_calculate, (myarg,), timeout_duration=5)