haproxy日志中有许多'cD'标志和许多非零Send-Q连接。怎么调试呢?

时间:2017-09-26 07:25:21

标签: tcp server haproxy

我使用haproxy作为服务器的tcp balancer。 haproxy和客户端之间有几百个非零的Send-Q连接。 haproxy日志中有很多'cD'标志。现在,许多服务器响应非常缓慢地到达客户端(超过10秒)。是由于客户端没有收到数据吗?或者haproxy服务器无法正常工作?或者haproxy服务器达到带宽限制?我该怎么做才能找到原因?

#455 non-zero Send-Q connection
ubuntu@ip-172-31-19-218:~$ netstat -atn|awk '{if($3>0) print $0}'|wc -l
455

#Top five Send-Q connections
ubuntu@ip-172-31-19-218:~$ netstat -atn|awk '{if($3>0) print $0}'|sort -k3nr|head -n 5
tcp        0  27292 172.31.19.218:12135      :47685    ESTABLISHED
tcp        0  22080 172.31.19.218:12135      :11817       ESTABLISHED
tcp        0  21886 172.31.19.218:12135      :12755     ESTABLISHED
tcp        0  21584 172.31.19.218:12135      :8753      ESTABLISHED

#many 'cD' flags in haproxy log    
ubuntu@ip-172-31-19-218:/var/log$ awk '{print $12}' haproxy.log | sort | uniq -c
      3 
   7525 --
   **4687 cD**
    526 CD
      1 /run/haproxy.pid
      3 SD

#some 'cD' flag logs
[27/Sep/2017:10:04:11.791] game nodes/s23 1/1/424425 34577 cD 4130/4130/4130/154/0 0/0
[27/Sep/2017:10:09:59.272] game nodes/s34 1/0/77777 3387 cD 4129/4129/4129/165/0 0/0
[27/Sep/2017:09:55:18.557] game nodes/s13 1/0/958654 84303 cD 4128/4128/4128/173/0 0/0
[27/Sep/2017:10:09:34.121] game nodes/s15 1/0/103309 3573 cD 4127/4127/4127/168/0 0/0

#haproxy config
ubuntu@ip-172-31-19-218:/var/log$ cat /etc/haproxy/haproxy.cfg 
global
    daemon
    maxconn 200000
    log 127.0.0.1 local0

defaults
    maxconn 200000
    timeout connect 5000  
    timeout client  60000  
    timeout server 60000 

listen game
    bind *:12135
    mode tcp
    option tcplog
    log global
    balance roundrobin
    default_backend nodes


backend nodes
    server s11 172.31.20.23:12137
    ....

0 个答案:

没有答案