Question

我使用Tornado作为Web服务器，用户可以通过前端页面提交任务，审核后他们可以启动提交的任务。在这种情况下，我想启动异步子进程来处理任务，所以我在请求处理程序中编写以下代码：

def task_handler():
    // handle task here

def start_a_process_for_task():
    p = multiprocessing.Process(target=task_handler,args=())
    p.start()
    return 0

我不关心子流程，只是为它启动流程并返回前端页面并告诉用户任务已启动。任务本身将在后台运行，并将其状态或结果记录到数据库，以便用户使用可以在以后的网页上查看。所以在这里我不想使用阻塞的p.join（），但是在任务完成后没有p.join（），子进程就变成了一个已经失效的进程，而且Tornado作为一个守护进程运行并且永远不会退出，失效的过程永远不会消失。

任何人都知道如何解决这个问题，谢谢。

Answer 1

The proper way to avoid defunct children is for the parent to gracefully clean up and close all resources of the exited child. This is normally done by join(), but if you want to avoid that, another approach could be to set up a global handler for the SIGCHLD signal on the parent.

SIGCHLD will be emitted whenever a child exits, and in the handler function you should either call Process.join() if you still have access to the process object, or even use os.wait() to "wait" for any child process to terminate and properly reap it. The wait time here should be 0 as you know for sure a child process has just exited. You will also be able to get the process' exit code / termination signal so it can also be a useful method to handle / log child process crashes.

Here's a quick example of doing this:

from __future__ import print_function

import os
import signal
import time
from multiprocessing import Process


def child_exited(sig, frame):
    pid, exitcode = os.wait()
    print("Child process {pid} exited with code {exitcode}".format(
        pid=pid, exitcode=exitcode
    ))


def worker():
    time.sleep(5)
    print("Process {pid} has completed it's work".format(pid=os.getpid()))


def parent():
    children = []

    # Comment out the following line to see zombie children
    signal.signal(signal.SIGCHLD, child_exited)

    for i in range(5):
        c = Process(target=worker)
        c.start()
        print("Parent forked out worker process {pid}".format(pid=c.pid))
        children.append(c)
        time.sleep(1)

    print("Forked out {c} workers, hit Ctrl+C to end...".format(c=len(children)))
    while True:
        time.sleep(5)


if __name__ == '__main__':
    parent()

One caveat is that I am not sure if this process works on non-Unix operating systems. It should work on Linux, Mac and other Unixes.

Answer 2

You need to join your subprocesses if you do not want to create zombies. You can do it in threads.

This is a dummy example. After 10 seconds, all your subprocesses are gone instead of being zombies. This launches a thread for every subprocess. Threads do not need to be joined or waited. A thread executes subprocess, joins it and then exits the thread as soon as the subprocess is completed.

import multiprocessing
import threading
from time import sleep

def task_processor():
    sleep(10)

class TaskProxy(threading.Thread):
    def __init__(self):
        super(TaskProxy, self).__init__()

    def run(self):
        p = multiprocessing.Process(target=task_processor,args=())
        p.start()
        p.join()

def task_handler():
    t = TaskProxy()
    t.daemon = True
    t.start()
    return

for _ in xrange(0,20):
    task_handler()

sleep(60)

多处理产生不存在的过程

2 个答案: