我使用haproxy作为服务器的tcp balancer。 haproxy和客户端之间有几百个非零的Send-Q连接。 haproxy日志中有很多'cD'标志。现在,许多服务器响应非常缓慢地到达客户端(超过10秒)。是由于客户端没有收到数据吗?或者haproxy服务器无法正常工作?或者haproxy服务器达到带宽限制?我该怎么做才能找到原因?
#455 non-zero Send-Q connection
ubuntu@ip-172-31-19-218:~$ netstat -atn|awk '{if($3>0) print $0}'|wc -l
455
#Top five Send-Q connections
ubuntu@ip-172-31-19-218:~$ netstat -atn|awk '{if($3>0) print $0}'|sort -k3nr|head -n 5
tcp 0 27292 172.31.19.218:12135 :47685 ESTABLISHED
tcp 0 22080 172.31.19.218:12135 :11817 ESTABLISHED
tcp 0 21886 172.31.19.218:12135 :12755 ESTABLISHED
tcp 0 21584 172.31.19.218:12135 :8753 ESTABLISHED
#many 'cD' flags in haproxy log
ubuntu@ip-172-31-19-218:/var/log$ awk '{print $12}' haproxy.log | sort | uniq -c
3
7525 --
**4687 cD**
526 CD
1 /run/haproxy.pid
3 SD
#some 'cD' flag logs
[27/Sep/2017:10:04:11.791] game nodes/s23 1/1/424425 34577 cD 4130/4130/4130/154/0 0/0
[27/Sep/2017:10:09:59.272] game nodes/s34 1/0/77777 3387 cD 4129/4129/4129/165/0 0/0
[27/Sep/2017:09:55:18.557] game nodes/s13 1/0/958654 84303 cD 4128/4128/4128/173/0 0/0
[27/Sep/2017:10:09:34.121] game nodes/s15 1/0/103309 3573 cD 4127/4127/4127/168/0 0/0
#haproxy config
ubuntu@ip-172-31-19-218:/var/log$ cat /etc/haproxy/haproxy.cfg
global
daemon
maxconn 200000
log 127.0.0.1 local0
defaults
maxconn 200000
timeout connect 5000
timeout client 60000
timeout server 60000
listen game
bind *:12135
mode tcp
option tcplog
log global
balance roundrobin
default_backend nodes
backend nodes
server s11 172.31.20.23:12137
....