haproxy - 不重用连接

时间:2015-08-28 18:55:02

标签: haproxy spray keep-alive

我在backend1 loadbalancer后面设置了12个后端节点(名为backend12HAProxy)。如果我理解HAProxy默认模式它应该使用http持久连接 - 问题是在我的情况下,基于我使用netstat工具看到的内容似乎无法正常工作。

下面是针对netstat后面的单个后端节点执行简单loadtest执行期间HAProxy命令的片段(另一个设置为MAINT状态)。我网络中的负载均衡器IP是10.254.1.39

ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0    171 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED on (0,18/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0    171 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED on (0,20/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED on (0,20/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp       88      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED on (0,20/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58492       ESTABLISHED off (0.00/0/0)
tcp        0    170 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED on (0,19/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58492       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED on (0,21/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58492       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0    171 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED on (0,20/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED on (0,19/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:58492       ESTABLISHED on (0,21/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0    170 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED on (0,20/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58492       ESTABLISHED off (0.00/0/0)
tcp        0    170 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED on (0,18/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED on (0,21/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58492       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED on (0,22/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:58272       ESTABLISHED off (0.00/0/0)
tcp        0    170 10.254.1.12:9091        10.254.1.39:58188       ESTABLISHED on (0,18/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58424       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58492       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58620       ESTABLISHED off (0.00/0/0)
tcp        0    170 10.254.1.12:9091        10.254.1.39:58643       ESTABLISHED on (0,19/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:58333       ESTABLISHED off (0.00/0/0)

我可以看到,随着时间的推移,更多的连接被打开,但旧的连接被保留并被重用。同样的事情是,如果我将backend1添加到MAINT并仅启用backend2

ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39458       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0    171 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED on (0,21/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39458       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED on (0,19/0/0)
tcp        0    171 10.254.1.13:9091        10.254.1.39:39458       ESTABLISHED on (0,20/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.13:9091        10.254.1.39:39458       ESTABLISHED on (0,19/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39526       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:

tcp        0      0 10.254.1.13:9091        10.254.1.39:39610       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39458       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39526       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ 
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:39610       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39307       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39671       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39374       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39458       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:39526       ESTABLISHED off (0.00/0/0)

当我将两个后端(以及更多)放到READY时,问题就出现了。

backend1的输出:

ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:59570       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:59574       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:59709       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:59706       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:60209       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:60212       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:60212       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:60826       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:60813       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:60831       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:60833       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:33498       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:33502       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:33500       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:33504       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:33507       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:34042       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:34039       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:34032       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:34036       ESTABLISHED on (0,18/0/0)
tcp        0    170 10.254.1.12:9091        10.254.1.39:34036       ESTABLISHED on (0,18/0/0)
tcp       96      0 10.254.1.12:9091        10.254.1.39:34045       ESTABLISHED off (0.00/0/0)
ubuntu@backend1:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.12:9091        10.254.1.39:38181       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38150       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38167       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38163       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38179       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38170       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38130       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38172       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38139       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38176       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:35450       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38158       ESTABLISHED off (0.00/0/0)
tcp        0    171 10.254.1.12:9091        10.254.1.39:38137       ESTABLISHED on (0,20/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38177       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38144       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38125       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38144       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.12:9091        10.254.1.39:38125       ESTABLISHED off (0.00/0/0)

backend2

ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:52464       ESTABLISHED off (0.00/0/0)
tcp        0    170 10.254.1.13:9091        10.254.1.39:52466       ESTABLISHED on (0,20/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:52628       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:52631       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:53127       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:53138       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:53137       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:53134       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:53819       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:53828       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:53826       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:53821       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0    170 10.254.1.13:9091        10.254.1.39:54736       ESTABLISHED on (0,18/0/0)
tcp        0    171 10.254.1.13:9091        10.254.1.39:54728       ESTABLISHED on (0,18/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:54711       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:54738       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0    170 10.254.1.13:9091        10.254.1.39:55342       ESTABLISHED on (0,18/0/0)
tcp        0    171 10.254.1.13:9091        10.254.1.39:55344       ESTABLISHED on (0,20/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:55340       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:55339       ESTABLISHED off (0.00/0/0)
ubuntu@backend2:~$ netstat -tao | grep 39:
tcp        0      0 10.254.1.13:9091        10.254.1.39:59507       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59502       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59489       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59491       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59471       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59497       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59493       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59509       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59457       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59479       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59511       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59470       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59505       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59465       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59488       ESTABLISHED off (0.00/0/0)
tcp        0      0 10.254.1.13:9091        10.254.1.39:59513       ESTABLISHED off (0.00/0/0)

查看负载均衡器端(10.254.1.39)上的端口号,它们正在更改,这意味着连接已关闭并再次重新打开。问题是 - 为什么?我假设如果在单节点情况下重用连接,我的后端服务器(spray-canhttp://spray.io/)能够使用http持久连接。

这是我的HAProxy配置:

global
    log 127.0.0.1 local0
    log 127.0.0.1 local1 notice
    #log loghost    local0 info
    maxconn 4096
    #chroot /usr/share/haproxy
    user haproxy
    group haproxy
    daemon
    #debug
    #quiet

defaults
    log global
    mode    http
    option  httplog
    option  dontlognull
    retries 3
    option redispatch
    maxconn 4096
    contimeout  5000
    clitimeout  50000
    srvtimeout  50000

frontend lb
    bind *:8080
    acl read_model_acl path_beg /check
    use_backend read_models if read_model_acl

backend read_models
   balance roundrobin
   server read-model1 10.254.1.12:9091 check 
   server read-model2 10.254.1.13:9091 check
   #more servers

0 个答案:

没有答案