Question

在Python GUI（PyGTK）中，我启动一个进程（使用多处理）。这个过程需要很长时间（约20分钟）才能完成。当过程完成后，我想清理它（提取结果并加入过程）。我怎么知道这个过程何时结束？

我的同事在父进程中建议了一个繁忙的循环，用于检查子进程是否已完成。当然有更好的方法。

在Unix中，当进程分叉时，a signal handler is called from within the parent process when the child process has finished。但我在Python中看不到类似的东西。我错过了什么吗？

如何从父进程中观察到子进程的结束？（当然，我不想调用Process.join（），因为它会冻结GUI界面。）

这个问题不仅限于多处理：我在多线程方面遇到了完全相同的问题。

Answer 1

我认为作为制作python多平台的一部分，像SIGCHLD这样的简单事情必须自己完成。同意，当你想要做的就是知道孩子何时完成时，这是一项更多的工作，但这真的不是那么痛苦。考虑以下使用子进程来完成工作，两个multiprocessing.Event实例，以及一个检查子进程是否完成的线程：

import threading
from multiprocessing import Process, Event
from time import sleep

def childsPlay(event):
    print "Child started"
    for i in range(3):
        print "Child is playing..."
        sleep(1)
    print "Child done"
    event.set()

def checkChild(event, killEvent):
    event.wait()
    print "Child checked, and is done playing"
    if raw_input("Do again? y/n:") == "y":
        event.clear()
        t = threading.Thread(target=checkChild, args=(event, killEvent))
        t.start()
        p = Process(target=childsPlay, args=(event,))
        p.start()
    else:
        cleanChild()
        killEvent.set()

def cleanChild():
    print "Cleaning up the child..."

if __name__ == '__main__':
    event = Event()
    killEvent = Event()

    # process to do work
    p = Process(target=childsPlay, args=(event,))
    p.start()

    # thread to check on child process
    t = threading.Thread(target=checkChild, args=(event, killEvent))
    t.start()

    try:
        while not killEvent.is_set():
            print "GUI running..."
            sleep(1)
    except KeyboardInterrupt:
        print "Quitting..."
        exit(0)
    finally:
        print "Main done"

修改

加入所有创建的进程和线程是一种很好的做法，因为它有助于指示何时创建僵尸（永不完成）进程/线程。我已经改变了上面的代码，使得一个继承自threading.Thread的ChildChecker类。它的唯一目的是在一个单独的进程中启动一个作业，等待该进程完成，然后在一切完成时通知GUI。加入ChildChecker也将加入它“检查”的过程。现在，如果进程在5秒后没有加入，则线程将强制终止进程。输入“y”创建启动运行“endlessChildsPlay”的子进程，该进程必须显示强制终止。

import threading
from multiprocessing import Process, Event
from time import sleep

def childsPlay(event):
    print "Child started"
    for i in range(3):
        print "Child is playing..."
        sleep(1)
    print "Child done"
    event.set()

def endlessChildsPlay(event):
    print "Endless child started"
    while True:
        print "Endless child is playing..."
        sleep(1)
        event.set()
    print "Endless child done"

class ChildChecker(threading.Thread):
    def __init__(self, killEvent):
        super(ChildChecker, self).__init__()
        self.killEvent = killEvent
        self.event = Event()
        self.process = Process(target=childsPlay, args=(self.event,))

    def run(self):
        self.process.start()

        while not self.killEvent.is_set():
            self.event.wait()
            print "Child checked, and is done playing"
            if raw_input("Do again? y/n:") == "y":
                self.event.clear()
                self.process = Process(target=endlessChildsPlay, args=(self.event,))
                self.process.start()
            else:
                self.cleanChild()
                self.killEvent.set()

    def join(self):
        print "Joining child process"
        # Timeout on 5 seconds
        self.process.join(5)

        if self.process.is_alive():
            print "Child did not join!  Killing.."
            self.process.terminate()
        print "Joining ChildChecker thread"
        super(ChildChecker, self).join()


    def cleanChild(self):
        print "Cleaning up the child..."

if __name__ == '__main__':
    killEvent = Event()
    # thread to check on child process
    t = ChildChecker(killEvent)
    t.start()

    try:
        while not killEvent.is_set():
            print "GUI running..."
            sleep(1)
    except KeyboardInterrupt:
        print "Quitting..."
        exit(0)
    finally:
        t.join()
        print "Main done"

Answer 2

您可以使用queue与子进程进行通信。您可以在其上粘贴中间结果，或者指示里程碑已被命中的消息（对于进度条）或仅指示该进程已准备好加入的消息。使用empty进行轮询非常简单快捷。

如果您真的只想知道它是否已完成，您可以观看流程的exitcode或投票is_alive()。

Answer 3

在努力寻找自己问题的答案时，我偶然发现了PyGTK的idle_add() function。这给了我以下可能性：

创建一个通过队列进行通信的新子进程。
创建一个侦听队列的侦听器线程，当子进程向侦听器发送一条消息说明它已完成时，侦听器调用idle_add（）来设置回调。
在下一次主循环期间，父进程将调用回调。
回调可以提取结果，加入子进程并加入侦听器线程。

这似乎是一种过于复杂的方式来重新创建Unix的call-callback-when-child-process-is-。

这必须是Python中GUI的常见问题。当然有一种标准模式可以解决这个问题吗？

Answer 4

这个答案很简单！（它只花了我天来解决它。）

结合PyGTK的idle_add（），您可以创建AutoJoiningThread。总代码是临界琐碎的：

class AutoJoiningThread(threading.Thread):
    def run(self):
        threading.Thread.run(self)
        gobject.idle_add(self.join)

如果您想做的不仅仅是加入（例如收集结果），那么您可以扩展上面的类以在完成时发出信号，如下例所示：

import threading
import time
import sys
import gobject
gobject.threads_init()

class Child:
    def __init__(self):
        self.result = None

    def play(self, count):
        print "Child starting to play."
        for i in range(count):
            print "Child playing."
            time.sleep(1)
        print "Child finished playing."
        self.result = 42

    def get_result(self, obj):
        print "The result was "+str(self.result)

class AutoJoiningThread(threading.Thread, gobject.GObject):
    __gsignals__ = {
        'finished': (gobject.SIGNAL_RUN_LAST,
                     gobject.TYPE_NONE,
                     ())
        }

    def __init__(self, *args, **kwargs):
        threading.Thread.__init__(self, *args, **kwargs)
        gobject.GObject.__init__(self)

    def run(self):
        threading.Thread.run(self)
        gobject.idle_add(self.join)
        gobject.idle_add(self.emit, 'finished')

    def join(self):
        threading.Thread.join(self)
        print "Called Thread.join()"

if __name__ == '__main__':
    print "Creating child"
    child = Child()
    print "Creating thread"
    thread = AutoJoiningThread(target=child.play,
                               args=(3,))
    thread.connect('finished', child.get_result)
    print "Starting thread"
    thread.start()
    print "Running mainloop (Ctrl+C to exit)"
    mainloop = gobject.MainLoop()

    try:
        mainloop.run()
    except KeyboardInterrupt:
        print "Received KeyboardInterrupt.  Quiting."
        sys.exit()

    print "God knows how we got here.  Quiting."
    sys.exit()

上述示例的输出将取决于执行线程的顺序，但它类似于：

Creating child
Creating thread
Starting thread
Child starting to play.
 Child playing.
Running mainloop (Ctrl+C to exit)
Child playing.
Child playing.
Child finished playing.
Called Thread.join()
The result was 42
^CReceived KeyboardInterrupt.  Quiting.

不可能以相同的方式创建AutoJoiningProcess（因为我们不能跨两个不同的进程调用idle_add（）），但是我们可以使用AutoJoiningThread来获得我们想要的东西：

class AutoJoiningProcess(multiprocessing.Process):
    def start(self):
        thread = AutoJoiningThread(target=self.start_process)
        thread.start() # automatically joins

    def start_process(self):
        multiprocessing.Process.start(self)
        self.join()

此处演示AutoJoiningProcess是另一个例子：

import threading
import multiprocessing
import time
import sys
import gobject
gobject.threads_init()

class Child:
    def __init__(self):
        self.result = multiprocessing.Manager().list()

    def play(self, count):
        print "Child starting to play."
        for i in range(count):
            print "Child playing."
            time.sleep(1)
    print "Child finished playing."
        self.result.append(42)

    def get_result(self, obj):
        print "The result was "+str(self.result)

class AutoJoiningThread(threading.Thread, gobject.GObject):
    __gsignals__ = {
        'finished': (gobject.SIGNAL_RUN_LAST,
                     gobject.TYPE_NONE,
                     ())
    }

    def __init__(self, *args, **kwargs):
        threading.Thread.__init__(self, *args, **kwargs)
        gobject.GObject.__init__(self)

    def run(self):
        threading.Thread.run(self)
        gobject.idle_add(self.join)
        gobject.idle_add(self.emit, 'finished')

    def join(self):
        threading.Thread.join(self)
        print "Called Thread.join()"

class AutoJoiningProcess(multiprocessing.Process, gobject.GObject):
    __gsignals__ = {
        'finished': (gobject.SIGNAL_RUN_LAST,
                     gobject.TYPE_NONE,
                     ())
        }

    def __init__(self, *args, **kwargs):
        multiprocessing.Process.__init__(self, *args, **kwargs)
        gobject.GObject.__init__(self)

    def start(self):
        thread = AutoJoiningThread(target=self.start_process)
        thread.start()

    def start_process(self):
        multiprocessing.Process.start(self)
        self.join()
        gobject.idle_add(self.emit, 'finished')

    def join(self):
        multiprocessing.Process.join(self)
        print "Called Process.join()"

if __name__ == '__main__':
    print "Creating child"
    child = Child()
    print "Creating thread"
    process = AutoJoiningProcess(target=child.play,
                               args=(3,))
    process.connect('finished',child.get_result)
    print "Starting thread"
    process.start()
    print "Running mainloop (Ctrl+C to exit)"
    mainloop = gobject.MainLoop()

    try:
        mainloop.run()
    except KeyboardInterrupt:
        print "Received KeyboardInterrupt.  Quiting."
        sys.exit()

    print "God knows how we got here.  Quiting."
    sys.exit()

结果输出将与上面的示例非常相似，除了这次我们同时加入了进程并且它的加入了线程：

Creating child
Creating thread
Starting thread
Running mainloop (Ctrl+C to exit)
 Child starting to play.
Child playing.
Child playing.
Child playing.
Child finished playing.
Called Process.join()
The result was [42]
Called Thread.join()
^CReceived KeyboardInterrupt.  Quiting.

不幸的是：

由于使用了idle_add（），此解决方案依赖于gobject。 gobject由PyGTK使用。
这不是真正的父/子关系。如果其中一个线程由另一个线程启动，那么它仍将由运行mainloop的线程加入，而不是父线程。这个问题也适用于AutoJoiningProcess，除了我想象会抛出异常。

因此，为了使用这种方法，最好只在mainloop / GUI中创建线程/进程。

Answer 5

查看子进程模块：

http://docs.python.org/library/subprocess.html

import subprocess
let pipe = subprocess.Popen("ls -l", stdout=subprocess.PIPE)
allText = pipe.stdout.read()
pipe.wait()
retVal = pipe.returncode

在Python中，我如何知道进程何时完成？

5 个答案:

修改