我们正在将docker用于烧瓶应用程序,并且正在使用nginx-uwsgi映像tiangolo/uwsgi-nginx-flask。
Docker镜像对于大多数应用程序都可以正常工作,但是其中一个部署有时会出错。
实际错误是
2020/10/21 18:17:49 [error] 12#12: *142201 connect() to unix:///tmp/uwsgi.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 10.255.0.212, server: , request: "GET /healthcheck HTTP/1.0", upstream: "uwsgi://unix:///tmp/uwsgi.sock:"
错误说,uwsgi
套接字存在问题。当我检查到uwsgi套接字的连接时,我得到了带有101
标签的CONNECTING
连接。
root@eaf23a900022:/app# netstat -nap | grep uwsgi | grep CONNECTING | wc -l
101
root@eaf23a900022:/app# netstat -nap | grep uwsgi | grep CONNECTING
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
unix 2 [ ] STREAM CONNECTING 0 - /tmp/uwsgi.sock
由此看来,nginx
与uwsgi
建立了连接,然后它无法释放它。
同一图像中还有其他烧瓶应用程序正在运行,我检查了我获得了多少个CONNECTING
套接字,但是其他容器返回了ZERO
。
root@466d15de99c7:/app# ps aux | grep uwsgi
root 10 0.3 0.2 298340 44636 ? Sl Oct17 11:00 /usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
root 36 0.0 0.2 298340 42776 ? S Oct18 0:00 /usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
root 37 0.0 0.2 298340 43316 ? S Oct18 0:00 /usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
root 320 0.0 0.0 12848 960 pts/0 S+ 17:23 0:00 grep uwsgi
root@466d15de99c7:/app# kill -9 10 36 37
root@466d15de99c7:/app# ps aux | grep uwsgi
root 321 43.0 0.3 222368 49872 ? D 17:24 0:00 /usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
root 325 0.0 0.0 12848 960 pts/0 S+ 17:24 0:00 grep uwsgi
root@466d15de99c7:/app# ps aux | grep uwsgi
root 321 23.5 0.3 298476 52380 ? Sl 17:24 0:00 /usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
root 327 0.0 0.2 298476 43728 ? S 17:24 0:00 /usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
root 328 0.0 0.2 298476 43728 ? S 17:24 0:00 /usr/local/bin/uwsgi --ini /etc/uwsgi/uwsgi.ini
root 330 0.0 0.0 12848 960 pts/0 S+ 17:24 0:00 grep uwsgi
root@466d15de99c7:/app# netstat -nap | grep uwsgi | grep CONNECTING | wc -l
0
其中有一些NGINX
调整参数集。 printenv
的输出如下。
[nile2691@983195-dfw301web02 ~]$ docker exec -it b2cff9ac29a2 printenv
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=b2cff9ac29a2
TERM=xterm
API_PASS=XXXXX
API_USER=XXXX
APP_ENVIRONMENT=bleeding
BCAPI_URL=https://example.com
BSAPI_APP=RpcApi
BSAPI_DB_NAME=RPC_Bleeding
BSAPI_DB_URI=mongodb://username:mongodb.instance:26200/database?ssl=true
BSAPI_ES_LOGGING=True
BSAPI_REDIS_URI=redis://:password@redis.instance:6379/0
BSNOTIFY_ENV=staging
CELERY_BROKER_URL=amqp://username:password@rabbit.instance/vhostg
NGINX_WORKER_CONNECTIONS=2048
NGINX_WORKER_OPEN_FILES= 2048
NGINX_WORKER_PROCESSES=8
LANG=C.UTF-8
GPG_KEY=0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D
PYTHON_VERSION=3.7.4
PYTHON_PIP_VERSION=19.2.1
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/404c9418e33c5031b1a9ab623168b3e8a2ed8c88/get-pip.py
PYTHON_GET_PIP_SHA256=56bb63d3cf54e7444351256f72a60f575f6d8c7f1faacffae33167afc8e7609d
NGINX_VERSION=1.15.8-1~stretch
NJS_VERSION=1.15.8.0.2.7-1~stretch
UWSGI_INI=/app/uwsgi.ini
UWSGI_CHEAPER=2
UWSGI_PROCESSES=16
NGINX_MAX_UPLOAD=0
LISTEN_PORT=80
STATIC_URL=/static
STATIC_PATH=/app/static
STATIC_INDEX=0
PYTHONPATH=/app
HOME=/root
我们试图了解为什么套接字连接卡在CONNECTING
中,并且在请求满足后仍无法清除。
是什么使此套接字队列达到101
个计数。
到达100+
时开始提示套接字错误。
我们可以增加队列的大小,但是我们可以肯定,它将再次充满。