HTTP Server在嵌套线程方案中保持线程活动

时间:2018-08-26 19:22:56

标签: python python-3.x multithreading httpserver

我有一个HTTPServer实现,我正在准备监听来自API的回调。在测试中,这种暗示使最里面的线程保持活动状态。这是服务器:

import http
import uuid
from http import server

class Server(server.HTTPServer):

    RequestLog:list = []
    ErrorList:list = []
    Serve:bool = True

    def __init__(self, server_address, RequestHandlerClass):

        self.RequestLog = []
        self.ErrorList = []
        self.Serve:bool = True

        return super().__init__(server_address, RequestHandlerClass)

    def LogRequest(self, clientAddress, success, state, params:dict={}):
        """docstring"""

        uid = uuid.uuid1()
        logItem = {"RequestID" : uid,
                   "ClientAddress" : clientAddress,
                   "Success" : success,
                   "State" : state,
                   "Params" : params}

        self.RequestLog.append(logItem)

    def GetRequestItem(self, state):
        """docstring"""

        logItem = {}
        if self.RequestLog and len(self.RequestLog):
            logItem = [d for d in self.RequestLog if d["State"] == state][0]

        return logItem

    def service_actions(self):

        try:
            if not self.Serve:
                self.shutdown()
                self.server_close()
        except Exception as e:
            err = e
            raise e

        return super().service_actions()

    def handle_error(self, request, client_address):


        logItem = {"clientAddress" : client_address,
                   "success" : False,
                   "state" : None,
                   "params" : None}

        try:
            self.LogRequest(**logItem)
            x = request
        except Exception as e:
            self.shutdown()
            err = e
            raise e

        return super().handle_error(request, client_address)

因此,上述服务器实现所执行的操作是在ResquestLog:list中记录有关请求的信息,然后提供一种方法GetRequestItem,该方法可用于拉取已记录请求的存在。在测试中,我抛出了错误并使用handle_error()覆盖捕获了它。这是调用函数,它启动服务器,轮询请求,然后通过将其Server.Serve方法设置为False

来关闭服务器。
def AwaitCallback(self, server_class=Server,
                     handler_class=OAuthGrantRequestHandler):
        """docstring"""

        server_address = ("127.0.0.1", 8080)
        self.Httpd = server_class(server_address, handler_class)
        self.Httpd.timeout = 200
        t1 = threading.Thread(target=self.Httpd.serve_forever)

        try:
            t1.start()

            #poll for request result
            result = {}
            x = 0
            while x < self.Timeout:
                if len(self.Httpd.RequestLog) > 0:
                    break
                time.sleep(.5)

        finally:
            #Terminate Server
            if self.Httpd:
                self.Httpd.Serve = False
            if t1:
                t1.join()
    return

以上方法坚持进行t1.join()调用。检查self.Httpd对象挂起时,它告诉我服务器serve_forever()的循环已关闭,但线程在调用t1.is_alive()时仍显示其活动状态。发生什么了?我唯一能想到的是,在t1线程中调用self.shutdown()时,它真的产生了循环,而不是关闭循环并保持循环运行吗?有关关闭的文档仅表示shutdown() : Tell the serve_forever() loop to stop and wait until it does.不错,也很模糊。有任何想法吗?

修改1: How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?中建议的答案完全不同。他们建议使用更简单的实现来覆盖socketserver.BaseServer.serve_forever()循环的所有本机功能,而我试图正确使用本机实现。到目前为止,就我所知,我上面的工作代码示例应达到答案所暗示的相同目的,但子线程不会终止。因此这个问题。

1 个答案:

答案 0 :(得分:0)

我需要OP进行验证,但是这里的问题是http.server.HTTPServer是基于进程而不是线程的,并且可能创建了僵尸进程。实际上,forkingMixIn explicitly handles this issue

  

serve_forever(poll_interval = 0.5)

     

处理请求,直到明确的shutdown()请求为止。每poll_interval秒轮询一次关机。忽略超时属性。   它还会调用service_actions(),它可以由子类或   mixin提供特定于给定服务的操作。 例如,   ForkingMixIn类使用service_actions()清理僵尸子级   流程。

这涉及迭代所有active_children,并调用discard函数。您可以检查socketserver.ForkingMixIn.collect_children的源代码(请参阅文章底部)。最快的解决方案可能是使用ThreadingHTTPServer

  

类http.server.ThreadingHTTPServer(server_address,   RequestHandlerClass)

     

此类与HTTPServer相同,但是使用线程通过ThreadingMixIn处理请求。 这对处理网络很有用   浏览器预打开套接字,HTTPServer将在套接字上等待   无限期地。

如果您想使用常规服务器

您将需要创建和处理active_children,并实现类似于对service_actions进行ForkingMixIn调用的方法:

def collect_children(self):
    """Internal routine to wait for children that have exited."""
    if self.active_children is None:
        return

    # If we're above the max number of children, wait and reap them until
    # we go back below threshold. Note that we use waitpid(-1) below to be
    # able to collect children in size(<defunct children>) syscalls instead
    # of size(<children>): the downside is that this might reap children
    # which we didn't spawn, which is why we only resort to this when we're
    # above max_children.
    while len(self.active_children) >= self.max_children:
        try:
            pid, _ = os.waitpid(-1, 0)
            self.active_children.discard(pid)
        except ChildProcessError:
            # we don't have any children, we're done
            self.active_children.clear()
        except OSError:
            break

    # Now reap all defunct children.
    for pid in self.active_children.copy():
        try:
            pid, _ = os.waitpid(pid, os.WNOHANG)
            # if the child hasn't exited yet, pid will be 0 and ignored by
            # discard() below
            self.active_children.discard(pid)
        except ChildProcessError:
            # someone else reaped it
            self.active_children.discard(pid)
        except OSError:
            pass