基于socketserver的Python 3服务器的关闭挂起

时间:2018-01-15 15:05:05

标签: python python-3.x gevent socketserver

我正在使用Python 3中使用线程SocketServer的“简单”服务器。

为此我实施关闭会遇到很多麻烦。我在互联网上找到的代码和关机最初工作,但通过telnet从客户端发送一些命令后停止工作。一些调查告诉我它挂在threading._shutdown ...... threading._wait_for_tstate_lock但到目前为止这并不响铃。

我的研究告诉我,在不同的python版本中有大约42种不同的解决方案,框架等。到目前为止,我找不到python3的工作方法。例如。我爱 telnetsrv  (https://pypi.python.org/pypi/telnetsrv/0.4)for python 2.7(它使用来自gevent的greenlets)但是这个不适用于python 3.所以如果有一个更pythonic,std lib方法或可靠的工作,我很想听听它!

我目前的赌注是套接字服务器,但我还没弄清楚如何处理挂起的服务器。我删除了所有日志语句和大多数功能,因此我可以发布这个暴露问题的最小服务器:

# -*- coding: utf-8 -*-
import socketserver
import threading

SERVER = None


def shutdown_cmd(request):
    global SERVER
    request.send(bytes('server shutdown requested\n', 'utf-8'))
    request.close()
    SERVER.shutdown()
    print('after shutdown!!')
    #SERVER.server_close()


class service(socketserver.BaseRequestHandler):
    def handle(self):
        while True:
            try:
                msg = str(self.request.recv(1024).strip(), 'utf-8')
                if msg == 'shutdown':
                    shutdown_cmd(msg, self.request)
                else:
                    self.request.send(bytes("You said '{}'\n".format(msg), "utf-8"))
            except Exception as e:
                pass


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def run():
    global SERVER
    SERVER = ThreadedTCPServer(('', 1520), service)
    server_thread = threading.Thread(target=SERVER.serve_forever)
    server_thread.daemon = True
    server_thread.start()
    input("Press enter to shutdown")
    SERVER.shutdown()


if __name__ == '__main__':
    run()

能够从处理程序中停止服务器也很棒(请参阅shutdown_cmd)

4 个答案:

答案 0 :(得分:3)

shutdown()按预期工作,服务器已停止接受新连接,但python仍在等待活动线程终止。

默认情况下,socketserver.ThreadingMixIn将创建新线程来处理传入连接,默认情况下,这些线程是非守护程序线程,因此python将等待所有活动的非守护程序线程终止。

当然,您可以使服务器生成守护程序线程,然后python不会等待:

  

ThreadingMixIn类定义了一个属性daemon_threads,它指示服务器是否应该等待线程终止。如果您希望线程自动运行,则应该显式设置标志;默认值为False,这意味着在ThreadingMixIn创建的所有线程都退出之前,Python不会退出。

class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    daemon_threads = True

但这不是理想的解决方案,你应该检查为什么线程永远不会终止,通常,服务器应该在没有新数据或客户端关闭连接时停止处理连接:

import socketserver
import threading


shutdown_evt = threading.Event()


class service(socketserver.BaseRequestHandler):
    def handle(self):
        self.request.setblocking(False)
        while True:
            try:
                msg = self.request.recv(1024)
                if msg == b'shutdown':
                    shutdown_evt.set()
                    break
                elif msg:
                    self.request.send(b'you said: ' + msg)
                if shutdown_evt.wait(0.1):
                    break
            except Exception as e:
                break


class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    pass


def run():
    SERVER = ThreadedTCPServer(('127.0.0.1', 10000), service)
    server_thread = threading.Thread(target=SERVER.serve_forever)
    server_thread.daemon = True
    server_thread.start()
    input("Press enter to shutdown")
    shutdown_evt.set()
    SERVER.shutdown()


if __name__ == '__main__':
    run()

答案 1 :(得分:1)

我尝试了两种解决方案来实现在Linux和Windows上运行在Python 3上的tcp服务器(我尝试过Windows 7):

  • 使用socketserver(我的问题) - 关闭无法正常工作
  • 使用asyncio(发布了答案) - 在Windows上不起作用

这两种解决方案均基于网络上的搜索结果。最后,我不得不放弃寻找经过验证的解决方案的想法,因为我找不到一个。因此,我实施了自己的解决方案(基于gevent)。我在这里发布是因为我希望其他人能够避免以我的方式进行捣乱。

# -*- coding: utf-8 -*-
from gevent.server import StreamServer
from gevent.pool import Pool


class EchoServer(StreamServer):

    def __init__(self, listener, handle=None, spawn='default'):
        StreamServer.__init__(self, listener, handle=handle, spawn=spawn)

    def handle(self, socket, address):
        print('New connection from %s:%s' % address[:2])
        socket.sendall(b'Welcome to the echo server! Type quit to exit.\r\n')

        # using a makefile because we want to use readline()
        rfileobj = socket.makefile(mode='rb')
        while True:
            line = rfileobj.readline()
            if not line:
                print("client disconnected")
                break
            if line.strip().lower() == b'quit':
                print("client quit")
                break
            if line.strip().lower() == b'shutdown':
                print("client initiated server shutdown")
                self.stop()
                break
            socket.sendall(line)
            print("echoed %r" % line.decode().strip())
        rfileobj.close()


srv = EchoServer(('', 1520), spawn=Pool(20))
srv.serve_forever()

答案 2 :(得分:0)

经过更多研究后,我发现了一个使用 asyncio

的样本
# -*- coding: utf-8 -*-
import asyncio

# after further research I found this relevant europython talk:
#     https://www.youtube.com/watch?v=pi49aiLBas8
# * protocols and transport are useful if you do not have tons of socket based code
# * event loop pushes data in
# * transport used to push data back to the client
# found decent sample in book by wrox "professional python"


class ServerProtocol(asyncio.Protocol):
    def connection_made(self, transport):
        self.transport = transport
        self.write('Welcome')

    def connection_lost(self, exc):
        self.transport = None

    def data_received(self, data):
        if not data or data == '':
            return
        message = data.decode('ascii')
        command = message.strip().split(' ')[0].lower()
        args = message.strip().split(' ')[1:]

        #sanity check
        if not hasattr(self, 'command_%s' % command):
            self.write('Invalid command: %s' % command)
            return

        # run command
        try:
            return getattr(self, 'command_%s' % command)(*args)
        except Exception as ex:
            self.write('Error: %s' % str(ex))

    def write(self, msg):
        self.transport.write((msg + '\n').encode('ascii', 'ignore'))

    def command_shutdown(self):
        self.write('Okay. shutting down')
        raise KeyboardInterrupt

    def command_bye(self):
        self.write('bye then!')
        self.transport.close()
        self.transport = None


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    coro = loop.create_server(ServerProtocol, '127.0.0.1', 8023)
    asyncio.async(coro)
    try:
        loop.run_forever()
    except KeyboardInterrupt:
        pass

据我所知,这是进行此类网络编程的最有用方法。如果需要,可以使用与uvloop(https://magic.io/blog/uvloop-blazing-fast-python-networking/)相同的代码来提高性能。

答案 3 :(得分:0)

关闭服务器的另一种方法是通过为serve_forever调用创建进程/线程。

server_forever启动后,只需等待自定义标志触发并在服务器上使用server_close,然后终止该进程。

streaming_server = StreamingServer(('', 8000), StreamingHandler)

FLAG_KEEP_ALIVE.value = True

process_serve_forever = Process(target=streaming_server.serve_forever)
process_serve_forever.start()

while FLAG_KEEP_ALIVE.value:
    pass

streaming_server.server_close()
process_serve_forever.terminate()