Docker Swarm负载平衡或网状路由问题

时间:2019-06-27 17:35:43

标签: docker nginx docker-swarm docker-networking kong

我有一个配置为docker swarm的docker节点。

我创建了一个名为main_net的覆盖网络,该网络使用加密功能。

我有多个容器连接到该网络。其中之一称为kong并运行kong。我有一个名为saas_thumbsum_0_0_13的服务-该服务在其中运行着一个容器。

Kong被配置为saas_thumbsum_0_0_13的反向代理,并使用地址saas_thumbsum_0_0_13。今天早晨,当我运行名为saas_thumbsum_0_0_12的服务时,此设置有效,Kong毫无问题地向http://saas_thumbsum_0_0_12:80/public发送了请求。

我现在已经创建了另一个服务0_0_13并将其连接到网络,但是kong无法访问http://saas_thumbsum_0_0_13:80/public

我已登录kong容器的控制台并运行以下命令:

/ # nslookup saas_thumbsum_0_0_13
nslookup: can't resolve '(null)': Name does not resolve

Name:      saas_thumbsum_0_0_13
Address 1: 10.0.0.39
/ # nslookup tasks.saas_thumbsum_0_0_13
nslookup: can't resolve '(null)': Name does not resolve

Name:      tasks.saas_thumbsum_0_0_13
Address 1: 10.0.0.40 d4c3034462d1.main_net

因此该服务解析为10.0.0.39,我猜这是负载均衡的IP,并且该服务具有一个IP为10.0.0.40的任务。到目前为止还可以。 现在,我尝试从kong容器中ping两个IP:

/ # ping 10.0.0.39
PING 10.0.0.39 (10.0.0.39): 56 data bytes
64 bytes from 10.0.0.39: seq=0 ttl=64 time=0.089 ms
64 bytes from 10.0.0.39: seq=1 ttl=64 time=0.083 ms
^C
--- 10.0.0.39 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.083/0.086/0.089 ms
/ # ping 10.0.0.40
PING 10.0.0.40 (10.0.0.40): 56 data bytes
64 bytes from 10.0.0.40: seq=0 ttl=64 time=0.146 ms
64 bytes from 10.0.0.40: seq=1 ttl=64 time=0.091 ms
^C
--- 10.0.0.40 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.091/0.118/0.146 ms

我可以成功ping通这两个IP地址。对我来说,这似乎很好,因为从我的初学者水平来看,他们应该将负载均衡到同一端点,并且结果不应有差异。

现在,如果我尝试从负载均衡的IP获取网页,则它将失败:

/ # wget 10.0.0.39/public/web/frontend
Connecting to 10.0.0.39 (10.0.0.39:80)
wget: can't connect to remote host (10.0.0.39): Connection refused

与我使用服务名称相同

/ # wget saas_thumbsum_0_0_13/public/web/frontend
Connecting to saas_thumbsum_0_0_13 (10.0.0.39:80)
wget: can't connect to remote host (10.0.0.39): Connection refused

有无http://

但是如果我转到任务ip,它就可以工作:

/ # wget 10.0.0.40/public/web/frontend
Connecting to 10.0.0.40 (10.0.0.40:80)
Connecting to 10.0.0.40 (10.0.0.40:80)
frontend             100% |*******************************************************|  1769   0:00:00 ETA

但是我应该使用服务端点而不是任务端点!

如果地址是负载均衡的,则它们都应到达相同的目标,因此结果应相同。

目标容器正在运行Nginx,我已经检查了那里的配置文件,但没有看到任何类型的源IP限制。不知道我该怎么做,因为IP地址每次都会不同!

所以这个问题坏了,wget saas_thumbsum_0_0_13 / public / web / frontend应该给我该页面,但是它不起作用。

如果有人可以发现我的错误或给我任何调试指针,将不胜感激。

一些其他有用的信息:

robert@metcaac4:~$ docker --version
Docker version 18.09.3, build 774a1f4
robert@metcaac4:~$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
ac295a4f901c        bridge              bridge              local
5aa2eea87b33        docker_gwbridge     bridge              local
3744b0f39195        host                host                local
lffazc510t7z        ingress             overlay             swarm
r8z2bb9idtwt        main_net            overlay             swarm
0ad4a8efa4cc        none                null                local

robert@metcaac4:~$ docker network inspect main_net
[
    {
        "Name": "main_net",
        "Id": "r8z2bb9idtwtyfr20e6u1p2f8",
        "Created": "2019-03-07T20:24:38.553471996Z",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.0.0/24",
                    "Gateway": "10.0.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": true,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "2b3b0d5311f23b084a2aa5282f451acb2c6d636b5ceb11926a46ee0a62e3035a": {
                "Name": "code_site2_code_site2.1.bqcsxy73pczsurit07lxmmoru",
                "EndpointID": "c9974c99e83338aa07f17eda56342757bce6a247105eb8fa4afab1b71f8d0366",
                "MacAddress": "02:42:0a:00:00:b7",
                "IPv4Address": "10.0.0.183/24",
                "IPv6Address": ""
            },
            "2ed56f6b2b06cbdeed62037cf10e40683b2960b2d53866e22a4080fa7a50a563": {
                "Name": "saas_user_management_0_0_66.1.0092odmxl4er803jhcrad7xhb",
                "EndpointID": "90f6cc3368c1fd0e5ff0be990f7cd0b1a854584eea3cdac3a80c0d535aa94c43",
                "MacAddress": "02:42:0a:00:00:f5",
                "IPv4Address": "10.0.0.245/24",
                "IPv6Address": ""
            },
            "7059b7eb266a6ad159fab1606b176c23d70f91841fb2635e5e163afa44f78e32": {
                "Name": "kong_kong.1.dr5afpgi5me23p96kub2qmjaq",
                "EndpointID": "0bfc5ece35f5278596b05232cbc5f88deef9c9a6d132d6f54b82b4c7c7e4d4fe",
                "MacAddress": "02:42:0a:00:00:b8",
                "IPv4Address": "10.0.0.184/24",
                "IPv6Address": ""
            },
            "7e0153b62eab821019b8f0757c842de367a61025e2848c0548c3cb7bd28d2df0": {
                "Name": "code_site2_code_site2_nginx.1.ijxkoaxzrly1b0dkt2ysdot3r",
                "EndpointID": "d776003ee4a98acd9cb1df2d3fe2861f4c8aca9c655a01dac9b371aebc4a6b2f",
                "MacAddress": "02:42:0a:00:00:27",
                "IPv4Address": "10.0.0.39/24",
                "IPv6Address": ""
            },
            "9cebbcf75312ce906786d722c38c30c85a84ec5b4efbe46656388f49fa50f942": {
                "Name": "saas_user_management_0_0_50.1.9xkwovfqejqerzi5co2mfb5l4",
                "EndpointID": "28cfcbd20723b272f787f1ecea7603b65f1bd48df8dcb89474087d3f5ae77ed7",
                "MacAddress": "02:42:0a:00:00:21",
                "IPv4Address": "10.0.0.33/24",
                "IPv6Address": ""
            },
            "a44f86449e19ca3a6fc170c3ed997156f0d3db66fece23883c1a3f844714e504": {
                "Name": "kong_dbb.1.l4d1xzqtr4t0thjnf39twfo0j",
                "EndpointID": "a410b8db987fd7eb261865c27dc84905bdfab273ccf20906c7b5a1ba0cbd1ab8",
                "MacAddress": "02:42:0a:00:00:ad",
                "IPv4Address": "10.0.0.173/24",
                "IPv6Address": ""
            },
            "d4c3034462d106496da5df3449d1da26dd50bcf40e4b954aea0b64a33a88d954": {
                "Name": "saas_thumbsum_0_0_13.1.of39j50gs0d5376nqgtsszx7m",
                "EndpointID": "a10e51cc60c559f9631a0fb883ca00a654bd1c815cc0a4730222be27847a802f",
                "MacAddress": "02:42:0a:00:00:28",
                "IPv4Address": "10.0.0.40/24",
                "IPv6Address": ""
            },
            "deda51bfbee097093ccb3e5be7ea11aa0648bdc74381f1552fe102b8646343cf": {
                "Name": "dockerregistry_dockerregistry.1.rty0eoy404m870f6qkpla9c9z",
                "EndpointID": "ff64d38ae4f932b7567cb0b5417df0cb8eef12e46572636a40d59da37511eff9",
                "MacAddress": "02:42:0a:00:00:ae",
                "IPv4Address": "10.0.0.174/24",
                "IPv6Address": ""
            },
            "e117e90a37cacbdbfc2768e2c77512b1c967e268718884f0372d63c1f6b79cbd": {
                "Name": "code_site2_code_site2_nginx.1.dihdw4g5n8i20pbb7nfwwi6ud",
                "EndpointID": "620d3c5de8169a380cb3c2e2caa91e18c91a1d4cf244900e5ea4912941e9c4c8",
                "MacAddress": "02:42:0a:00:00:ac",
                "IPv4Address": "10.0.0.172/24",
                "IPv6Address": ""
            },
            "lb-main_net": {
                "Name": "main_net-endpoint",
                "EndpointID": "d4b3747a3701b05f716f47295132939bed6f68c566fbf5ab50b70071e2d5cd0d",
                "MacAddress": "02:42:0a:00:00:02",
                "IPv4Address": "10.0.0.2/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.driver.overlay.vxlanid_list": "4097",
            "encrypted": ""
        },
        "Labels": {},
        "Peers": [
            {
                "Name": "1a2559b7bdb0",
                "IP": "78.31.105.225"
            }
        ]
    }
]



robert@metcaac4:~$ docker service ls
ID                  NAME                            MODE                REPLICAS            IMAGE                                         PORTS
miqimzjmh0qe        code_site2_code_site2           replicated          1/1                 metcarob/phpfpm_for_drupal:0.0.6              
bgixg86fhaqs        code_site2_code_site2_nginx     replicated          1/1                 nginx:1.13.0-alpine                           
0d2267ul4iv4        dockerregistry_dockerregistry   replicated          1/1                 registry:2                                    *:5000->5000/tcp
4tn5xqbss3i5        kong_dbb                        replicated          1/1                 postgres:9.6                                  
u9xo3prf53v2        kong_kong                       replicated          1/1                 kong:1.1.2                                    *:80->8000/tcp, *:443->8443/tcp
kgmhthn93qwy        saas_thumbsum_0_0_13            replicated          1/1                 metcarob/saas_thumbsum:0.0.13                 
zhyna2ktbgw4        saas_user_management_0_0_66     replicated          1/1                 metcarob/saas_user_management_system:0.0.66

我尝试删除并重新创建saas_thumbsum服务,但这没有什么区别。 (仅更改了该IP)

我可以提供saas_thumbsum_0_0_13 nginx配置,但是自从工作版本以来这没有改变,我不认为这是问题所在。

感谢您的帮助!

更新001-额外信息

我的saas_user_management_0_0_66服务具有完全相同的设置。这没有问题。

robert@metcaac4:~$ docker exec -it kong_kong.1.dr5afpgi5me23p96kub2qmjaq /bin/sh
/ # nslookup saas_user_management_system_0_0_66
nslookup: can't resolve '(null)': Name does not resolve

nslookup: can't resolve 'saas_user_management_system_0_0_66': Name does not resolve
/ # nslookup saas_user_management_0_0_66
nslookup: can't resolve '(null)': Name does not resolve

Name:      saas_user_management_0_0_66
Address 1: 10.0.0.244
/ # nslookup tasks.saas_user_management_0_0_66
nslookup: can't resolve '(null)': Name does not resolve

Name:      tasks.saas_user_management_0_0_66
Address 1: 10.0.0.245 saas_user_management_0_0_66.1.0092odmxl4er803jhcrad7xhb.main_net
/ # pint 10.0.0.244
/bin/sh: pint: not found
/ # ping 10.0.0.244
PING 10.0.0.244 (10.0.0.244): 56 data bytes
64 bytes from 10.0.0.244: seq=0 ttl=64 time=0.082 ms
64 bytes from 10.0.0.244: seq=1 ttl=64 time=0.113 ms
^C
--- 10.0.0.244 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.082/0.097/0.113 ms
/ # ping 10.0.0.245
PING 10.0.0.245 (10.0.0.245): 56 data bytes
64 bytes from 10.0.0.245: seq=0 ttl=64 time=0.177 ms
64 bytes from 10.0.0.245: seq=1 ttl=64 time=0.112 ms
^C
--- 10.0.0.245 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.112/0.144/0.177 ms
/ # wget 10.0.0.245
Connecting to 10.0.0.245 (10.0.0.245:80)
wget: server returned error: HTTP/1.1 404 Not Found
/ # wget 10.0.0.244
Connecting to 10.0.0.244 (10.0.0.244:80)
wget: server returned error: HTTP/1.1 404 Not Found

在上面的wget中,10.0.0.245和10.0.0.244给出了与服务器完全相同的404找不到响应。我不知道为什么saas_thumbsum拒绝连接。

1 个答案:

答案 0 :(得分:0)

我发现了自己的问题。在这里发表这篇文章可能会很有用,因为其他人可能会遇到同样的事情。

我发了消息,并使用以下命令启动了容器:

docker service create --name saas_thumbsum_0_0_13 --network main_net metcarob/:0.0.13

应该在什么时候

docker service create --name saas_thumbsum_0_0_13 --network main_net metcarob/saas_thumbsum:0.0.13

(由于在我的自动部署过程中使用了环境变量)

我不知道为什么没有容器就启动了服务,我也不知道为什么一个IP有效而另一个IP不起作用。 另外,我能够对容器执行docker exec命令,因此必须已在运行某些东西。

无论如何,现在已修复!