Python源代码：
import time
import os
import signal
import threading
import datetime
import uwsgi
to_be_killed = {}
def print_still_alive():
now = time.time()
still_alive_pids = set(w['pid'] for w in uwsgi.workers() if w['pid'] > 0)
print datetime.datetime.now(), ('STILL ALIVE: %r' % {k: int(now - v) for k, v in
to_be_killed.iteritems() if k in still_alive_pids})
class PeriodicTimer(threading._Timer):
def run(self):
while not self.finished.is_set():
self.finished.wait(self.interval)
if not self.finished.is_set():
self.function(*self.args, **self.kwargs)
def cycle():
print_still_alive()
for w in uwsgi.workers():
if w['pid'] <= 0:
continue
pid = w['pid']
to_be_killed.setdefault(pid, time.time())
os.kill(pid, signal.SIGHUP)
print datetime.datetime.now(), 'Sent HUP to %s' % pid
if uwsgi.masterpid() == os.getpid():
timer = PeriodicTimer(5, cycle, ())
timer.setDaemon(True)
timer.start()
def application(env, start_response):
start_response('200 OK', [('Content-Type', 'text/html')])
return ['Hello World %s: %s' % (os.getpid(), time.time())]
使用以下命令运行：
uwsgi --http :8003 --wsgi simple_th:application --honour-stdin --master --workers 5 --py-auto-reload=1 --enable-threads
$ uwsgi --version
2.0.8
的Linux：
$ uname -a
Linux xxx 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
问题是经过一段时间后工作人员停止响应HUP信号。在日志中看起来像这样：
$ uwsgi --http :8003 --wsgi simple_th:application --honour-stdin --master --workers 5 --py-auto-reload=1 --enable-threads
*** Starting uWSGI 2.0.8 (64bit) on [Wed Nov 19 15:43:35 2014] ***
compiled with version: 4.8.2 on 14 November 2014 19:44:37
os: Linux-3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014
nodename: dmugtasimov-ThinkPad-T520
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /home/dmugtasimov/gitrep/apigateway/tmp
detected binary path: /home/dmugtasimov/gitrep/apigateway/ENV/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
your processes number limit is 63268
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :8003 fd 4
uwsgi socket 0 bound to TCP address 127.0.0.1:36063 (port auto-assigned) fd 3
Python version: 2.7.6 (default, Mar 22 2014, 23:03:41) [GCC 4.8.2]
Python main interpreter initialized at 0x1f0f350
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 436608 bytes (426 KB) for 5 cores
*** Operational MODE: preforking ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x1f0f350 pid: 21265 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 21265)
spawned uWSGI worker 1 (pid: 21267, cores: 1)
spawned uWSGI worker 2 (pid: 21268, cores: 1)
spawned uWSGI worker 3 (pid: 21269, cores: 1)
spawned uWSGI worker 4 (pid: 21270, cores: 1)
spawned uWSGI worker 5 (pid: 21271, cores: 1)
spawned uWSGI http 1 (pid: 21272)
Python auto-reloader enabled
2014-11-19 15:43:40.553574 STILL ALIVE: {}
2014-11-19 15:43:40.554045 Sent HUP to 21267
Gracefully killing worker 1 (pid: 21267)...
2014-11-19 15:43:40.554127 Sent HUP to 21268
Gracefully killing worker 2 (pid: 21268)...
Gracefully killing worker 3 (pid: 21269)...
2014-11-19 15:43:40.555841 Sent HUP to 21269
2014-11-19 15:43:40.555936 Sent HUP to 21270
Gracefully killing worker 5 (pid: 21271)...
Gracefully killing worker 4 (pid: 21270)...
2014-11-19 15:43:40.560523 Sent HUP to 21271
Respawned uWSGI worker 1 (new pid: 21281)
Respawned uWSGI worker 2 (new pid: 21282)
Respawned uWSGI worker 3 (new pid: 21283)
Respawned uWSGI worker 4 (new pid: 21284)
Respawned uWSGI worker 5 (new pid: 21285)
Python auto-reloader enabled
2014-11-19 15:43:45.561159 STILL ALIVE: {}
Gracefully killing worker 1 (pid: 21281)...
2014-11-19 15:43:45.561491 Sent HUP to 21281
Gracefully killing worker 2 (pid: 21282)...
2014-11-19 15:43:45.562548 Sent HUP to 21282
Gracefully killing worker 3 (pid: 21283)...
2014-11-19 15:43:45.562621 Sent HUP to 21283
2014-11-19 15:43:45.562665 Sent HUP to 21284
Gracefully killing worker 5 (pid: 21285)...
Gracefully killing worker 4 (pid: 21284)...
2014-11-19 15:43:45.569720 Sent HUP to 21285
Respawned uWSGI worker 1 (new pid: 21292)
Respawned uWSGI worker 2 (new pid: 21293)
Respawned uWSGI worker 3 (new pid: 21294)
Respawned uWSGI worker 4 (new pid: 21295)
Respawned uWSGI worker 5 (new pid: 21296)
Python auto-reloader enabled
2014-11-19 15:43:50.570273 STILL ALIVE: {}
2014-11-19 15:43:50.570476 Sent HUP to 21292
Gracefully killing worker 1 (pid: 21292)...
2014-11-19 15:43:50.570516 Sent HUP to 21293
Gracefully killing worker 2 (pid: 21293)...
2014-11-19 15:43:50.570572 Sent HUP to 21294
2014-11-19 15:43:50.570622 Sent HUP to 21295
Gracefully killing worker 4 (pid: 21295)...
2014-11-19 15:43:50.570673 Sent HUP to 21296
Gracefully killing worker 5 (pid: 21296)...
Gracefully killing worker 3 (pid: 21294)...
Respawned uWSGI worker 1 (new pid: 21304)
Respawned uWSGI worker 2 (new pid: 21305)
Respawned uWSGI worker 3 (new pid: 21306)
Respawned uWSGI worker 4 (new pid: 21307)
Respawned uWSGI worker 5 (new pid: 21308)
Python auto-reloader enabled
2014-11-19 15:43:55.571036 STILL ALIVE: {}
Gracefully killing worker 1 (pid: 21304)...
2014-11-19 15:43:55.571167 Sent HUP to 21304
Gracefully killing worker 2 (pid: 21305)...
2014-11-19 15:43:55.571200 Sent HUP to 21305
2014-11-19 15:43:55.571232 Sent HUP to 21306
2014-11-19 15:43:55.571260 Sent HUP to 21307
2014-11-19 15:43:55.571290 Sent HUP to 21308
Gracefully killing worker 3 (pid: 21306)...
Gracefully killing worker 4 (pid: 21307)...
Gracefully killing worker 5 (pid: 21308)...
Respawned uWSGI worker 1 (new pid: 21317)
Python auto-reloader enabled
Respawned uWSGI worker 2 (new pid: 21322)
Respawned uWSGI worker 3 (new pid: 21323)
Respawned uWSGI worker 4 (new pid: 21324)
Respawned uWSGI worker 5 (new pid: 21325)
2014-11-19 15:44:00.571999 STILL ALIVE: {}
Gracefully killing worker 1 (pid: 21317)...
2014-11-19 15:44:00.572323 Sent HUP to 21317
2014-11-19 15:44:00.572408 Sent HUP to 21322
Gracefully killing worker 2 (pid: 21322)...
2014-11-19 15:44:00.572461 Sent HUP to 21323
2014-11-19 15:44:00.572516 Sent HUP to 21324
Gracefully killing worker 5 (pid: 21325)...
2014-11-19 15:44:00.572568 Sent HUP to 21325
Gracefully killing worker 3 (pid: 21323)...
Gracefully killing worker 4 (pid: 21324)...
Respawned uWSGI worker 2 (new pid: 21331)
Respawned uWSGI worker 3 (new pid: 21332)
Respawned uWSGI worker 5 (new pid: 21333)
Respawned uWSGI worker 1 (new pid: 21337)
Respawned uWSGI worker 4 (new pid: 21338)
Python auto-reloader enabled
2014-11-19 15:44:05.572873 STILL ALIVE: {}
Gracefully killing worker 1 (pid: 21337)...
2014-11-19 15:44:05.575816 Sent HUP to 21337
2014-11-19 15:44:05.575888 Sent HUP to 21331
Gracefully killing worker 3 (pid: 21332)...
2014-11-19 15:44:05.579080 Sent HUP to 21332
2014-11-19 15:44:05.579159 Sent HUP to 21338
2014-11-19 15:44:05.579189 Sent HUP to 21333
Gracefully killing worker 5 (pid: 21333)...
Gracefully killing worker 2 (pid: 21331)...
Gracefully killing worker 4 (pid: 21338)...
Respawned uWSGI worker 2 (new pid: 21342)
Respawned uWSGI worker 3 (new pid: 21343)
Respawned uWSGI worker 5 (new pid: 21344)
Respawned uWSGI worker 1 (new pid: 21345)
Respawned uWSGI worker 4 (new pid: 21346)
Python auto-reloader enabled
2014-11-19 15:44:10.579536 STILL ALIVE: {}
Gracefully killing worker 1 (pid: 21345)...
2014-11-19 15:44:10.582839 Sent HUP to 21345
2014-11-19 15:44:10.583022 Sent HUP to 21342
2014-11-19 15:44:10.583111 Sent HUP to 21343
2014-11-19 15:44:10.583188 Sent HUP to 21346
Gracefully killing worker 4 (pid: 21346)...
Gracefully killing worker 5 (pid: 21344)...
2014-11-19 15:44:10.586615 Sent HUP to 21344
Gracefully killing worker 3 (pid: 21343)...
Gracefully killing worker 2 (pid: 21342)...
Respawned uWSGI worker 2 (new pid: 21353)
Respawned uWSGI worker 3 (new pid: 21354)
Respawned uWSGI worker 5 (new pid: 21355)
Respawned uWSGI worker 1 (new pid: 21356)
Respawned uWSGI worker 4 (new pid: 21357)
Python auto-reloader enabled
2014-11-19 15:44:15.587064 STILL ALIVE: {}
Gracefully killing worker 1 (pid: 21356)...
2014-11-19 15:44:15.587251 Sent HUP to 21356
2014-11-19 15:44:15.587290 Sent HUP to 21353
2014-11-19 15:44:15.587324 Sent HUP to 21354
Gracefully killing worker 3 (pid: 21354)...
2014-11-19 15:44:15.587356 Sent HUP to 21357
2014-11-19 15:44:15.587390 Sent HUP to 21355
Gracefully killing worker 4 (pid: 21357)...
Gracefully killing worker 2 (pid: 21353)...
Gracefully killing worker 5 (pid: 21355)...
Respawned uWSGI worker 2 (new pid: 21367)
Respawned uWSGI worker 3 (new pid: 21368)
Respawned uWSGI worker 5 (new pid: 21369)
Respawned uWSGI worker 1 (new pid: 21370)
Respawned uWSGI worker 4 (new pid: 21371)
Python auto-reloader enabled
2014-11-19 15:44:20.588180 STILL ALIVE: {}
Gracefully killing worker 1 (pid: 21370)...
2014-11-19 15:44:20.588577 Sent HUP to 21370
Gracefully killing worker 2 (pid: 21367)...
2014-11-19 15:44:20.588658 Sent HUP to 21367
2014-11-19 15:44:20.588704 Sent HUP to 21368
Gracefully killing worker 3 (pid: 21368)...
2014-11-19 15:44:20.588749 Sent HUP to 21371
Gracefully killing worker 5 (pid: 21369)...
2014-11-19 15:44:20.588797 Sent HUP to 21369
gateway "uWSGI http 1" has been buried (pid: 21272)
...gracefully killing workers...
respawned uWSGI http 1 (pid: 21380)
worker 2 killed successfully (pid: 21367)
Respawned uWSGI worker 2 (new pid: 21381)
worker 3 killed successfully (pid: 21368)
Respawned uWSGI worker 3 (new pid: 21382)
worker 5 killed successfully (pid: 21369)
Respawned uWSGI worker 5 (new pid: 21383)
worker 1 killed successfully (pid: 21370)
Respawned uWSGI worker 1 (new pid: 21384)
Python auto-reloader enabled
2014-11-19 15:44:25.589164 STILL ALIVE: {21371: 5}
2014-11-19 15:44:25.589327 Sent HUP to 21384
Gracefully killing worker 1 (pid: 21384)...
2014-11-19 15:44:25.589356 Sent HUP to 21381
2014-11-19 15:44:25.589381 Sent HUP to 21382
2014-11-19 15:44:25.589421 Sent HUP to 21371
2014-11-19 15:44:25.589450 Sent HUP to 21383
Gracefully killing worker 2 (pid: 21381)...
Gracefully killing worker 3 (pid: 21382)...
Gracefully killing worker 5 (pid: 21383)...
Respawned uWSGI worker 2 (new pid: 21390)
Respawned uWSGI worker 3 (new pid: 21391)
Respawned uWSGI worker 5 (new pid: 21392)
Respawned uWSGI worker 1 (new pid: 21393)
Python auto-reloader enabled
2014-11-19 15:44:30.589732 STILL ALIVE: {21371: 10}
我做了5次试验。重现所需的时间因秒而异：5,60,20,50,35。因此，需要一些时间才能看到问题。
为什么会这样？
P.S。 另一个问题，也许相关。有时工人会尝试处理SIGINT / SIGQUIT信号（Linux终端中的Ctrl + C）
^CThe process id is 21265
SIGINT/SIGQUIT received...killing workers...
The process id is 21371
SIGINT/SIGQUIT received...killing workers...
gateway "uWSGI http 1" has been buried (pid: 21380)
gateway "uWSGI http 1" has been buried (pid: 21380)
worker 2 buried after 1 seconds
worker 3 buried after 1 seconds
worker 5 buried after 1 seconds
worker 1 buried after 1 seconds
2014-11-19 15:45:15.629073 STILL ALIVE: {21371: 55}
2014-11-19 15:45:15.629531 Sent HUP to 21371
注意：我在“SIGINT / SIGQUIT收到......杀死工人......”之前插入了“进程ID为％d”的printf，以查看哪个进程获得信号
21371是一个工人的PID，在Ctrl + C之前很久就停止响应SIGHUP
有时工作者尝试处理SIGINT / SIGQUIT而不会对SIGHUP没有响应，所以我认为这些问题只是部分相关。