Question

如何强制某个函数及其所有子进程在Linux上超时？

例如，如何在{10}之后强制完成multiprocessed_func：

import time

def multiprocessed_func(seconds):
    # Assume this a long running function which uses
    # multiprocessing internally and returns None.
    time.sleep(seconds)

try:
    multiprocessed_func(600)
except:
    print('took too long')

Answer 1

借用psutil文档，我们可以检查当前进程并在给定时间后终止或终止所有子进程。

def terminate_children(grace_period):
    procs = psutil.Process().children()
    for p in procs:
        p.terminate()
    gone, still_alive = psutil.wait_procs(procs, timeout=grace_period)
    for p in still_alive:
        p.kill()
    raise TimeoutError

try:
    multiprocessed_func(long_run=600)
    time.sleep(10)  # then timeout
    terminate_children(grace_period=2)
except TimeoutError:
    print('timed out')
    pass

完整示例：

import multiprocessing
import time
import psutil

def slow_worker(long_run):
    print('started')
    time.sleep(long_run)
    print('finished')

def multiprocessed_func(long_run):
    jobs = []
    for i in range(5):
        p = multiprocessing.Process(target=slow_worker, args=(long_run,))
        jobs.append(p)
        p.start()
        print('starting', p.pid)

def on_terminate(proc):
    print('terminating {}, exit code {}'.format(proc, proc.returncode))

def terminate_children(grace_period):
    procs = psutil.Process().children()
    for p in procs:
        p.terminate()
    gone, still_alive = psutil.wait_procs(procs, timeout=grace_period, 
                                          callback=on_terminate)
    for p in still_alive:
        p.kill()
    raise TimeoutError

try:
    multiprocessed_func(long_run=600)
    time.sleep(10)
    terminate_children(grace_period=2)
except TimeoutError:
    print('timed out')
    pass

如果终止所有当前进程中的子进程过多，因为当前进程中还有其他需要保留的多进程方法，那么我们可以包装 multiprocessed_func 在另一个过程中。

def safe_run(timeout, grace_period):
    try:
        multiprocessed_func(long_run=600)
        time.sleep(timeout)
        terminate_children(grace_period)
    except TimeoutError:
        pass

timeout, grace_period = 10, 2
p = multiprocessing.Process(target=safe_run, args=(timeout, grace_period,))
p.start()
p.join()
p.terminate()
time.sleep(2)
if p.is_alive():
    p.kill()

如何在Linux上超时运行函数及其所有子进程？

1 个答案: