在WSGIREF.simple_server旁边运行的线程内调用POpen会导致os.fork死锁

时间:2012-10-05 18:21:46

标签: python rest fork wsgi popen

此问题与A Django Related Question

有关

我正在使用wsgiref.simple_server运行restlite WSGI应用程序。我有这个设置,以便在serve_forever()方法获取调用之前初始化一些对象。最相关的这些课程。

import Queue
import threading
import Deployer
import ParallelRunner
import sys
import subprocess

class DeployManager:
def __init__(self):
    self.inputQueue = Queue.Queue() 
    self.workerThread = DeployManagerWorkerThread(self.inputQueue)
    self.workerThread.start()   
def addDeployJob(self, appList):
    self.inputQueue.put(appList)  #make sure this handles the queue being full
def stopWorker(self):
    self.workerThread.running = False
def __del__(self):
    self.stopWorker()

class DeployManagerWorkerThread(threading.Thread):
def __init__(self, Queue):
    super(DeployManagerWorkerThread, self).__init__()
    self.queue = Queue
    self.running = True
    self.deployer = Deployer.Deployer()
    self.runner = ParallelRunner.ParallelRunner()

def run(self):
    while self.running:
        try:
            appList = self.queue.get(timeout = 10) #This blocks until something is in the queue
            sys.stdout.write('Got deployment job\n')
                            command = "ssh " + server.sshUsername + "@" + server.hostname + "" + " -i " + server.sshPrivateKeyPath + r" 'bash -s' < " + pathToScript 
                            self.process = subprocess.Popen(command,shell=True ,stdin=subprocess.PIPE, stdout=subprocess.PIPE)
            output = process.communicate()[0]
            sys.stdout.write(output + '\n')
        except Queue.Empty:
            pass
    sys.stdout.write('DeployManagerWorkerThread exiting\n')

restlite请求设置如下

@restlite.resource
def deployAll():
def POST(request, entity):
    GlobalDeployManager.addDeployJob(GlobalAppList) #TODO should runners be configurable at start up
    #print output                                               #or on a run by run basis
    return 'Job added'
return locals()

这会将一个条目放入队列中,然后workerThread会抓取并开始处理。然而,对POpen的调用总是挂起我进入这个并且它似乎挂起来调用os.fork()将“冻结”服务器。当线程进入POpen命令时,主线程位于接受新请求的行。我相信服务器正在使用epoll。如果队列中有多个作业,我可以在控制台上按control-C,服务器正在运行,主线程将退出(不在等待请求的行),然后线程将能够运行按预期工作然后关闭。有什么想法吗?

1 个答案:

答案 0 :(得分:0)

我最终想出来了......我需要在run方法中创建线程对象。我不小心忘记了从主线程调用了 init 方法,然后新线程本身从不执行 init