Question

我正在使用ProcessPoolExecutor来生成子进程。目前我正在尝试在KeyboardInterrupt / Ctrl + C上优雅地退出脚本。

我正在创建有2名工人的游泳池并提交5个期货。在中断时我试图取消尚未执行的每个未来。如果我在执行前两个期货期间中断，则池只能取消两个期货，这意味着三个期货正在运行。但我只有两名工人，每个过程运行5秒钟。我未来的执行是什么或为什么？

import subprocess
from concurrent.futures import ProcessPoolExecutor
import signal
import sys


def executecommands(commands):
    # Replace signal handler of parent process, so child processes will ignore terminate signals
    original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN)
    processpool = ProcessPoolExecutor(1)
    # Restore original handler, so the parent process can handle terminate signals
    signal.signal(signal.SIGINT, original_sigint_handler)
    futures = []
    try:
        for command in commands:
            futures.append(processpool.submit(executecommand, command))

        processpool.shutdown()
        print("Executed commands without interruption")
    except KeyboardInterrupt:
        print("\nAttempting to cancel pending commands..")
        for future in futures:
            if future.cancel():
                print("Cancelled one command")
            else:
                print("One command was already running, couldn't cancel")
        print("Waiting for running processes to finish..")
        processpool.shutdown()
        print("Shutdown complete")
        sys.exit(0)


def executecommand(command):
    # create a subprocess and run it
    print("running command")
    process = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    print("finished command")
    return process

if __name__ == '__main__':
    print("lets go")
    commandlist = [['/bin/sleep', '5'], ['/bin/sleep', '5'], ['/bin/sleep', '5'], ['/bin/sleep', '5'], ['/bin/sleep', '5']]
    executecommands(commandlist)

Answer 1

这是一个CPython实现细节，但是你可以取消的唯一未来是那些不在＆＃34;呼叫队列＆＃34;中的未来。调用队列包含下一步要执行的所有期货。它的大小是max_workers + EXTRA_QUEUED_CALLS。（EXTRA_QUEUED_CALLS is currently set to 1。）

在您的情况下，当您的前两个期货开始执行时，呼叫队列将填充3下一个期货（max_workers为2，EXTRA_QUEUED_CALLS为1）。由于您总共只有5个期货，因此您无法取消任何期货。

如果你用2个工人填写10个期货的命令清单，你将可以取消最后五个期货：

lets go
running command
running command
^C
Attempting to cancel pending commands..
One command was already running, couldn't cancel
One command was already running, couldn't cancel
One command was already running, couldn't cancel
One command was already running, couldn't cancel
One command was already running, couldn't cancel
Cancelled one command
Cancelled one command
Cancelled one command
Cancelled one command
Cancelled one command
Waiting for running processes to finish..
running command
running command
finished command
finished command
running command
finished command
Shutdown complete

ProcessPoolExecutor锁定的期货超过了必要的数量

1 个答案: