Question

我在python中实现了一个简单的HTTP服务器，如下所示：

from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-Type', 'text/html')
        self.end_headers()
        self.wfile.write('Hello World!')

if __name__ == '__main__':
    try:
        server = HTTPServer(('', 8000), Handler)
        print 'Listening on 0.0.0.0:8000'
        server.serve_forever()
    except KeyboardInterrupt:
        print '^C, halting'
        server.socket.close()

然后我运行并发级别为50的apache bench：

ab -v 3 -c 50 -n 50000 http://localhost:8000/

我开始在apache bench中遇到这个错误：

apr_socket_recv：由对等方重置连接（104）

在HTTP服务器控制台上我看到了：

Exception happened during processing of request from ('127.0.0.1', 51545)
Traceback (most recent call last):

  File "/usr/lib/python2.6/SocketServer.py", line 281, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 307, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__
    self.handle()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "./test.py", line 12, in do_GET
    self.send_response(200)
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 384, in send_response
    self.send_header('Server', self.version_string())
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 390, in send_header
    self.wfile.write("%s: %s\r\n" % (keyword, value))
  File "/usr/lib/python2.6/socket.py", line 300, in write
    self.flush()
  File "/usr/lib/python2.6/socket.py", line 286, in flush
    self._sock.sendall(buffer)
error: [Errno 32] Broken pipe

另外我注意到HTTPServer变得非常迟缓

我试图通过这样做来增加HTTPServer的request_queue_size：

from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
class MyServer(HTTPServer):
    request_queue_size = 100
    timeout = None


class Handler(BaseHTTPRequestHandler):
    counter = 0

    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-Type', 'text/html')
        self.end_headers()
        self.wfile.write(str(self.counter))
        Handler.counter += 1

if __name__ == '__main__':
    try:
        server = MyServer(('', 8000), Handler)
        print 'Listening on 0.0.0.0:8000'
        server.serve_forever()
    except KeyboardInterrupt:
        print '^C, halting'
        server.socket.close()

但是我偶尔会在测试运行结束时得到破坏的管道错误。如果我将request_queue_size增加到一个巨大的数字，那么错误就会消失。我不明白为什么因为Apache工作台-c值为50且request_queue_size为100就足够了。为什么request_queue_size需要比Apache工作台并发设置大得多？即使前一个连接尚未完成，Apache工作台是否会启动更多连接，因此并发连接总数可能超过-c设置？

Answer 1

这是apachebench的问题，但不是httperf。我不确定他们处理连接的方式有何不同。

从默认值（5）增加到1000.我没有注意到内存中的任何大爆炸，性能大幅提升，同时消除了错误。

Apache bench和python BaseHTTPServer破坏管道异常

1 个答案: