按照教程https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/部署带有运河网络插件的单节点kubernetes。
# kubeadm init --pod-network-cidr 10.244.0.0/16 --kubernetes-version stable-1.9
kube-dns容器并非都在运行。
# kubectl -n kube-system get pod
NAME READY STATUS RESTARTS AGE
canal-mpzrt 3/3 Running 0 6h
etcd-gavin-k8s 1/1 Running 0 6h
kube-apiserver-gavin-k8s 1/1 Running 0 6h
kube-controller-manager-gavin-k8s 1/1 Running 0 6h
kube-dns-6f4fd4bdf-fc8pd 2/3 Running 0 53s
kube-proxy-vj2r9 1/1 Running 0 2h
kube-scheduler-gavin-k8s 1/1 Running 0 6h
kubectl -n kube-system logs kube-dns-6f4fd4bdf-fc8pd kubedns
I0425 08:40:41.303524 1 dns.go:48] version: 1.14.6-3-gc36cb11
I0425 08:40:41.304274 1 server.go:69] Using configuration read from directory: /kube-dns-config with period 10s
I0425 08:40:41.304308 1 server.go:112] FLAG: --alsologtostderr="false"
I0425 08:40:41.304316 1 server.go:112] FLAG: --config-dir="/kube-dns-config"
I0425 08:40:41.304326 1 server.go:112] FLAG: --config-map=""
I0425 08:40:41.304330 1 server.go:112] FLAG: --config-map-namespace="kube-system"
I0425 08:40:41.304334 1 server.go:112] FLAG: --config-period="10s"
I0425 08:40:41.304340 1 server.go:112] FLAG: --dns-bind-address="0.0.0.0"
I0425 08:40:41.304343 1 server.go:112] FLAG: --dns-port="10053"
I0425 08:40:41.304349 1 server.go:112] FLAG: --domain="cluster.local."
I0425 08:40:41.304354 1 server.go:112] FLAG: --federations=""
I0425 08:40:41.304359 1 server.go:112] FLAG: --healthz-port="8081"
I0425 08:40:41.304363 1 server.go:112] FLAG: --initial-sync-timeout="1m0s"
I0425 08:40:41.304367 1 server.go:112] FLAG: --kube-master-url=""
I0425 08:40:41.304372 1 server.go:112] FLAG: --kubecfg-file=""
I0425 08:40:41.304376 1 server.go:112] FLAG: --log-backtrace-at=":0"
I0425 08:40:41.304382 1 server.go:112] FLAG: --log-dir=""
I0425 08:40:41.304386 1 server.go:112] FLAG: --log-flush-frequency="5s"
I0425 08:40:41.304391 1 server.go:112] FLAG: --logtostderr="true"
I0425 08:40:41.304394 1 server.go:112] FLAG: --nameservers=""
I0425 08:40:41.304398 1 server.go:112] FLAG: --stderrthreshold="2"
I0425 08:40:41.304401 1 server.go:112] FLAG: --v="2"
I0425 08:40:41.304405 1 server.go:112] FLAG: --version="false"
I0425 08:40:41.304411 1 server.go:112] FLAG: --vmodule=""
I0425 08:40:41.304482 1 server.go:194] Starting SkyDNS server (0.0.0.0:10053)
I0425 08:40:41.304700 1 server.go:213] Skydns metrics enabled (/metrics:10055)
I0425 08:40:41.304709 1 dns.go:146] Starting endpointsController
I0425 08:40:41.304715 1 dns.go:149] Starting serviceController
I0425 08:40:41.308584 1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I0425 08:40:41.308603 1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
I0425 08:40:41.804866 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:42.304875 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:42.804873 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:43.304871 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:43.804868 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:44.304880 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:44.804873 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:45.304869 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:45.804863 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:46.304833 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:46.804868 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:47.304876 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0425 08:40:47.804878 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
我发现kube-dns失败的根本原因是pod中的容器无法访问我机器的物理ip。 主节点运行在192.168.80.167
# kubectl cluster-info
Kubernetes master is running at https://192.168.80.167:6443
KubeDNS is running at https://192.168.80.167:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
196.18.80.167是我机器上物理网桥的地址。
# ifconfig br0
br0 Link encap:Ethernet HWaddr 24:5E:BE:0C:C5:92
inet addr:192.168.80.167 Bcast:192.168.81.255 Mask:255.255.254.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4661901 errors:0 dropped:191628 overruns:0 frame:0
TX packets:317984 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1116345980 (1.0 GiB) TX bytes:56761158 (54.1 MiB)
# brctl show br0
bridge name bridge id STP enabled interfaces
br0 8000.245ebe0cc592 no eth0
kubedns容器无法访问我机器的物理网桥ip，然后就失败了。
# kubectl -n kube-system exec -it kube-dns-6f4fd4bdf-fc8pd --container kubedns -- sh
/ # ping 192.168.80.167
PING 192.168.80.167 (192.168.80.167): 56 data bytes
^C
--- 192.168.80.167 ping statistics ---
16 packets transmitted, 0 packets received, 100% packet loss
奇怪的是kubedns可以访问局域网中的其他机器。它无法访问仅运行pod的我的机器。
/ # ping 192.168.80.107
PING 192.168.80.107 (192.168.80.107): 56 data bytes
64 bytes from 192.168.80.107: seq=0 ttl=63 time=0.361 ms
64 bytes from 192.168.80.107: seq=1 ttl=63 time=0.342 ms
64 bytes from 192.168.80.107: seq=2 ttl=63 time=4.112 ms
^C
--- 192.168.80.107 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.342/1.605/4.112 ms
通过tcpdump分析网络流量，来自calic0b238d4ce2的流量不会转发到br0，因此没有人接听流量。
# tcpdump -i caliec0efa8668a -Q inout | grep ICMP
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on caliec0efa8668a, link-type EN10MB (Ethernet), capture size 262144 bytes
09:05:31.950671 IP 10.244.0.3 > Gavin-K8S: ICMP echo request, id 34560, seq 54, length 64
09:05:32.950733 IP 10.244.0.3 > Gavin-K8S: ICMP echo request, id 34560, seq 55, length 64
09:05:33.950794 IP 10.244.0.3 > Gavin-K8S: ICMP echo request, id 34560, seq 56, length 64
# tcpdump -i br0 -Q inout | grep ICMP
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
P.S：每个用户pod与kubedns的情况相同：pods无法访问运行它的节点，但是他们可以访问其他计算机。
在主机（主节点）上，检查路由表
# ip route show
default via 192.168.80.254 dev br0 proto static metric 100
10.0.3.0/24 dev lxcbr0 proto kernel scope link src 10.0.3.1
10.0.5.0/24 dev docker0 proto kernel scope link src 10.0.5.1 dead linkdown
10.244.0.4 dev calic0b238d4ce2 scope link
10.244.0.6 dev cali45026c409f9 scope link
10.244.0.7 dev caliec0efa8668a scope link
169.254.0.0/16 dev docker_gwbridge proto kernel scope link src 169.254.8.151
192.168.80.0/23 dev br0 proto kernel scope link src 192.168.80.167
# ip route get 192.168.80.167
local 192.168.80.167 dev lo src 192.168.80.167
cache <local>
在容器上，检查路由表
/ # ip route show
default via 169.254.1.1 dev eth0
169.254.1.1 dev eth0
/ # ip route get 192.168.80.167
192.168.80.167 via 169.254.1.1 dev eth0 src 10.244.0.7
iptable-save
的结果# Generated by iptables-save v1.6.0 on Wed Apr 25 21:25:22 2018
*raw
:PREROUTING ACCEPT [5988958:1384538104]
:OUTPUT ACCEPT [4321136:929267397]
:cali-OUTPUT - [0:0]
:cali-PREROUTING - [0:0]
:cali-failsafe-in - [0:0]
:cali-failsafe-out - [0:0]
:cali-from-host-endpoint - [0:0]
:cali-to-host-endpoint - [0:0]
-A PREROUTING -m comment --comment "cali:6gwbT8clXdHdC1b1" -j cali-PREROUTING
-A OUTPUT -m comment --comment "cali:tVnHkvAo15HuiPy0" -j cali-OUTPUT
-A cali-OUTPUT -m comment --comment "cali:WX1xZBEtmbS0Rhjs" -j MARK --set-xmark 0x0/0xf000000
-A cali-OUTPUT -m comment --comment "cali:iE00ZyllJNXfrlg_" -j cali-to-host-endpoint
-A cali-OUTPUT -m comment --comment "cali:Asois4hxp1rUxwJS" -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-PREROUTING -m comment --comment "cali:zatSDPVUhhPCk6Iy" -j MARK --set-xmark 0x0/0xf000000
-A cali-PREROUTING -i cali+ -m comment --comment "cali:-ES4EW0vxFmM81t8" -j MARK --set-xmark 0x4000000/0x4000000
-A cali-PREROUTING -m comment --comment "cali:VE1J3S_1t9q8GAsm" -m mark --mark 0x0/0x4000000 -j cali-from-host-endpoint
-A cali-PREROUTING -m comment --comment "cali:VX8l4jKL9w89GXz5" -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-failsafe-in -p tcp -m comment --comment "cali:wWFQM43tJU7wwnFZ" -m multiport --dports 22 -j ACCEPT
-A cali-failsafe-in -p udp -m comment --comment "cali:LwNV--R8MjeUYacw" -m multiport --dports 68 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:73bZKoyDfOpFwC2T" -m multiport --dports 2379 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:QMFuWo6o-d9yOpNm" -m multiport --dports 2380 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:Kup7QkrsdmfGX0uL" -m multiport --dports 4001 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:xYYr5PEqDf_Pqfkv" -m multiport --dports 7001 -j ACCEPT
-A cali-failsafe-out -p udp -m comment --comment "cali:nbWBvu4OtudVY60Q" -m multiport --dports 53 -j ACCEPT
-A cali-failsafe-out -p udp -m comment --comment "cali:UxFu5cDK5En6dT3Y" -m multiport --dports 67 -j ACCEPT
COMMIT
# Completed on Wed Apr 25 21:25:22 2018
# Generated by iptables-save v1.6.0 on Wed Apr 25 21:25:22 2018
*nat
:PREROUTING ACCEPT [16:2103]
:INPUT ACCEPT [14:1981]
:OUTPUT ACCEPT [5:677]
:POSTROUTING ACCEPT [4:617]
:KUBE-MARK-DROP - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODEPORTS - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-SEP-JPEBCQ2YOSKQPXKG - [0:0]
:KUBE-SERVICES - [0:0]
:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
:SYSDOCKER - [0:0]
:SYSNAT - [0:0]
:VPNNAT - [0:0]
:cali-OUTPUT - [0:0]
:cali-POSTROUTING - [0:0]
:cali-PREROUTING - [0:0]
:cali-fip-dnat - [0:0]
:cali-fip-snat - [0:0]
:cali-nat-outgoing - [0:0]
-A PREROUTING -m comment --comment "cali:6gwbT8clXdHdC1b1" -j cali-PREROUTING
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -m comment --comment "cali:tVnHkvAo15HuiPy0" -j cali-OUTPUT
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A POSTROUTING -m comment --comment "cali:O3lYWMrLQYEMJtB5" -j cali-POSTROUTING
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A POSTROUTING -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
-A POSTROUTING -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE
-A POSTROUTING ! -s 10.244.0.0/16 -d 10.244.0.0/24 -j RETURN
-A POSTROUTING ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE
-A KUBE-MARK-DROP -j MARK --set-xmark 0x8000/0x8000
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE
-A KUBE-SEP-JPEBCQ2YOSKQPXKG -s 192.168.80.167/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-JPEBCQ2YOSKQPXKG -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-JPEBCQ2YOSKQPXKG --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 192.168.80.167:6443
-A KUBE-SERVICES ! -s 10.244.0.0/16 -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
-A KUBE-SERVICES ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 10800 --reap --name KUBE-SEP-JPEBCQ2YOSKQPXKG --mask 255.255.255.255 --rsource -j KUBE-SEP-JPEBCQ2YOSKQPXKG
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -j KUBE-SEP-JPEBCQ2YOSKQPXKG
-A cali-OUTPUT -m comment --comment "cali:GBTAv2p5CwevEyJm" -j cali-fip-dnat
-A cali-POSTROUTING -m comment --comment "cali:Z-c7XtVd2Bq7s_hA" -j cali-fip-snat
-A cali-POSTROUTING -m comment --comment "cali:nYKhEzDlr11Jccal" -j cali-nat-outgoing
-A cali-PREROUTING -m comment --comment "cali:r6XmIziWUJsdOK6Z" -j cali-fip-dnat
-A cali-nat-outgoing -m comment --comment "cali:Wd76s91357Uv7N3v" -m set --match-set cali4-masq-ipam-pools src -m set ! --match-set cali4-all-ipam-pools dst -j MASQUERADE
COMMIT
# Completed on Wed Apr 25 21:25:23 2018
# Generated by iptables-save v1.6.0 on Wed Apr 25 21:25:23 2018
*mangle
:PREROUTING ACCEPT [1727587:391808161]
:INPUT ACCEPT [5150922:1211808224]
:FORWARD ACCEPT [1062:89161]
:OUTPUT ACCEPT [4321182:929275109]
:POSTROUTING ACCEPT [4331603:931649202]
:VPNCUSSETMARK - [0:0]
:VPNDEFSETMARK - [0:0]
:cali-PREROUTING - [0:0]
:cali-failsafe-in - [0:0]
:cali-from-host-endpoint - [0:0]
-A PREROUTING -m comment --comment "cali:6gwbT8clXdHdC1b1" -j cali-PREROUTING
-A PREROUTING -j VPNCUSSETMARK
-A PREROUTING -m mark --mark 0x0/0xffff -j VPNDEFSETMARK
-A VPNCUSSETMARK -m set --match-set vpnbr0 src -j MARK --set-xmark 0x900/0xff00
-A VPNCUSSETMARK -m set --match-set vpndocker0 src -j MARK --set-xmark 0xa00/0xff00
-A VPNCUSSETMARK -m set --match-set vpnlxcbr0 src -j MARK --set-xmark 0xc00/0xff00
-A cali-PREROUTING -m comment --comment "cali:6BJqBjBC7crtA-7-" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A cali-PREROUTING -m comment --comment "cali:nE3PUa5RSRqBBvwx" -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-PREROUTING -i cali+ -m comment --comment "cali:qgFofvzQe6yJPouQ" -j ACCEPT
-A cali-PREROUTING -m comment --comment "cali:o178eO5vvpj8e65z" -j cali-from-host-endpoint
-A cali-PREROUTING -m comment --comment "cali:5TQcm-i_T8rVGEEa" -m comment --comment "Host endpoint policy accepted packet." -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-failsafe-in -p tcp -m comment --comment "cali:wWFQM43tJU7wwnFZ" -m multiport --dports 22 -j ACCEPT
-A cali-failsafe-in -p udp -m comment --comment "cali:LwNV--R8MjeUYacw" -m multiport --dports 68 -j ACCEPT
COMMIT
# Completed on Wed Apr 25 21:25:23 2018
# Generated by iptables-save v1.6.0 on Wed Apr 25 21:25:23 2018
*filter
:INPUT ACCEPT [3389:699050]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [2944:635600]
:DOCKER-USER - [0:0]
:KUBE-FIREWALL - [0:0]
:KUBE-FORWARD - [0:0]
:KUBE-SERVICES - [0:0]
:SYSDOCKER - [0:0]
:SYSDOCKER-ISOLATION - [0:0]
:cali-FORWARD - [0:0]
:cali-INPUT - [0:0]
:cali-OUTPUT - [0:0]
:cali-failsafe-in - [0:0]
:cali-failsafe-out - [0:0]
:cali-from-host-endpoint - [0:0]
:cali-from-wl-dispatch - [0:0]
:cali-fw-cali45026c409f9 - [0:0]
:cali-fw-calic0b238d4ce2 - [0:0]
:cali-fw-caliec0efa8668a - [0:0]
:cali-pri-k8s_ns.default - [0:0]
:cali-pri-k8s_ns.kube-system - [0:0]
:cali-pro-k8s_ns.default - [0:0]
:cali-pro-k8s_ns.kube-system - [0:0]
:cali-to-host-endpoint - [0:0]
:cali-to-wl-dispatch - [0:0]
:cali-tw-cali45026c409f9 - [0:0]
:cali-tw-calic0b238d4ce2 - [0:0]
:cali-tw-caliec0efa8668a - [0:0]
:cali-wl-to-host - [0:0]
-A INPUT -m comment --comment "cali:Cz_u1IQiXIMmKD4c" -j cali-INPUT
-A INPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A INPUT -j KUBE-FIREWALL
-A FORWARD -m comment --comment "cali:wUHhoiAYhphO9Mso" -j cali-FORWARD
-A FORWARD -m comment --comment "kubernetes forward rules" -j KUBE-FORWARD
-A FORWARD -s 10.244.0.0/16 -j ACCEPT
-A FORWARD -d 10.244.0.0/16 -j ACCEPT
-A FORWARD -i br0 -o caliec0efa8668a -j ACCEPT
-A FORWARD -i caliec0efa8668a -o br0 -j ACCEPT
-A OUTPUT -m comment --comment "cali:tVnHkvAo15HuiPy0" -j cali-OUTPUT
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -j KUBE-FIREWALL
-A DOCKER-USER -j RETURN
-A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT
-A KUBE-FORWARD -s 10.244.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod source rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A KUBE-FORWARD -d 10.244.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns has no endpoints" -m udp --dport 53 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp has no endpoints" -m tcp --dport 53 -j REJECT --reject-with icmp-port-unreachable
-A SYSDOCKER-ISOLATION -j RETURN
-A cali-FORWARD -i cali+ -m comment --comment "cali:X3vB2lGcBrfkYquC" -j cali-from-wl-dispatch
-A cali-FORWARD -o cali+ -m comment --comment "cali:UtJ9FnhBnFbyQMvU" -j cali-to-wl-dispatch
-A cali-FORWARD -i cali+ -m comment --comment "cali:Tt19HcSdA5YIGSsw" -j ACCEPT
-A cali-FORWARD -o cali+ -m comment --comment "cali:9LzfFCvnpC5_MYXm" -j ACCEPT
-A cali-FORWARD -m comment --comment "cali:7AofLLOqCM5j36rM" -j MARK --set-xmark 0x0/0xe000000
-A cali-FORWARD -m comment --comment "cali:QM1_joSl7tL76Az7" -m mark --mark 0x0/0x1000000 -j cali-from-host-endpoint
-A cali-FORWARD -m comment --comment "cali:C1QSog3bk0AykjAO" -j cali-to-host-endpoint
-A cali-FORWARD -m comment --comment "cali:DmFiPAmzcisqZcvo" -m comment --comment "Host endpoint policy accepted packet." -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-INPUT -m comment --comment "cali:i7okJZpS8VxaJB3n" -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-INPUT -i cali+ -m comment --comment "cali:JaoDb6CLdcGw8g0Y" -g cali-wl-to-host
-A cali-INPUT -m comment --comment "cali:c5eKVW2VdKQ_LiSM" -j MARK --set-xmark 0x0/0xf000000
-A cali-INPUT -m comment --comment "cali:hwQKYSlSCkpE_9uN" -j cali-from-host-endpoint
-A cali-INPUT -m comment --comment "cali:ttp8-serzKCP-bKZ" -m comment --comment "Host endpoint policy accepted packet." -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-OUTPUT -m comment --comment "cali:YQSSJIsRcHjFbXaI" -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-OUTPUT -o cali+ -m comment --comment "cali:KRjBsKsBcFBYKCEw" -j RETURN
-A cali-OUTPUT -m comment --comment "cali:3VKAQBcyUUW5kS_j" -j MARK --set-xmark 0x0/0xf000000
-A cali-OUTPUT -m comment --comment "cali:Z1mBCSH1XHM6qq0k" -j cali-to-host-endpoint
-A cali-OUTPUT -m comment --comment "cali:N0jyWt2RfBedKw3L" -m comment --comment "Host endpoint policy accepted packet." -m mark --mark 0x1000000/0x1000000 -j ACCEPT
-A cali-failsafe-in -p tcp -m comment --comment "cali:wWFQM43tJU7wwnFZ" -m multiport --dports 22 -j ACCEPT
-A cali-failsafe-in -p udp -m comment --comment "cali:LwNV--R8MjeUYacw" -m multiport --dports 68 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:73bZKoyDfOpFwC2T" -m multiport --dports 2379 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:QMFuWo6o-d9yOpNm" -m multiport --dports 2380 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:Kup7QkrsdmfGX0uL" -m multiport --dports 4001 -j ACCEPT
-A cali-failsafe-out -p tcp -m comment --comment "cali:xYYr5PEqDf_Pqfkv" -m multiport --dports 7001 -j ACCEPT
-A cali-failsafe-out -p udp -m comment --comment "cali:nbWBvu4OtudVY60Q" -m multiport --dports 53 -j ACCEPT
-A cali-failsafe-out -p udp -m comment --comment "cali:UxFu5cDK5En6dT3Y" -m multiport --dports 67 -j ACCEPT
-A cali-from-wl-dispatch -i cali45026c409f9 -m comment --comment "cali:QTLwRyKNiscc-kE7" -g cali-fw-cali45026c409f9
-A cali-from-wl-dispatch -i calic0b238d4ce2 -m comment --comment "cali:7mRUmkMzCXKDHDzk" -g cali-fw-calic0b238d4ce2
-A cali-from-wl-dispatch -i caliec0efa8668a -m comment --comment "cali:vI_cBpGlZQpakzSQ" -g cali-fw-caliec0efa8668a
-A cali-from-wl-dispatch -m comment --comment "cali:y5WqyrGI7OWfnqNM" -m comment --comment "Unknown interface" -j DROP
-A cali-fw-cali45026c409f9 -m comment --comment "cali:OTJIDsP3TegJFYqm" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A cali-fw-cali45026c409f9 -m comment --comment "cali:uvhYBVFYvBcMfF1E" -m conntrack --ctstate INVALID -j DROP
-A cali-fw-cali45026c409f9 -m comment --comment "cali:N9Pier8knvEySzpb" -j MARK --set-xmark 0x0/0x1000000
-A cali-fw-cali45026c409f9 -m comment --comment "cali:6ctr2BZXeRQITWs2" -j cali-pro-k8s_ns.kube-system
-A cali-fw-cali45026c409f9 -m comment --comment "cali:Juq9dxqhxLUhudVk" -m comment --comment "Return if profile accepted" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-fw-cali45026c409f9 -m comment --comment "cali:o7CTzqIS9bu5DymV" -m comment --comment "Drop if no profiles matched" -j DROP
-A cali-fw-calic0b238d4ce2 -m comment --comment "cali:2dB9gQ0XK7ky-okg" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A cali-fw-calic0b238d4ce2 -m comment --comment "cali:ywcP6SMI-Q-GlUyW" -m conntrack --ctstate INVALID -j DROP
-A cali-fw-calic0b238d4ce2 -m comment --comment "cali:wroMotnj-PmPY-A1" -j MARK --set-xmark 0x0/0x1000000
-A cali-fw-calic0b238d4ce2 -m comment --comment "cali:nOL8WwmNyRPNDCRb" -j cali-pro-k8s_ns.default
-A cali-fw-calic0b238d4ce2 -m comment --comment "cali:r1XYAvTJ5M_XMUux" -m comment --comment "Return if profile accepted" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-fw-calic0b238d4ce2 -m comment --comment "cali:8-iYoFbdlSboxtvI" -m comment --comment "Drop if no profiles matched" -j DROP
-A cali-fw-caliec0efa8668a -m comment --comment "cali:NvFOTdFzvt46kQfQ" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A cali-fw-caliec0efa8668a -m comment --comment "cali:jxl0wYR8pO3dsQLg" -m conntrack --ctstate INVALID -j DROP
-A cali-fw-caliec0efa8668a -m comment --comment "cali:VlVHHstfJPnNr3LI" -j MARK --set-xmark 0x0/0x1000000
-A cali-fw-caliec0efa8668a -m comment --comment "cali:DlqVod2qRMSGS4t4" -j cali-pro-k8s_ns.default
-A cali-fw-caliec0efa8668a -m comment --comment "cali:LluPSlt2p5-XuwUs" -m comment --comment "Return if profile accepted" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-fw-caliec0efa8668a -m comment --comment "cali:23YDqnq73LBpscup" -m comment --comment "Drop if no profiles matched" -j DROP
-A cali-pri-k8s_ns.default -m comment --comment "cali:6MWuUqsVPzpSgE3L" -j MARK --set-xmark 0x1000000/0x1000000
-A cali-pri-k8s_ns.default -m comment --comment "cali:UGCdoOXoPRcONGv8" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-pri-k8s_ns.kube-system -m comment --comment "cali:plMTf6GGo5FLt-zw" -j MARK --set-xmark 0x1000000/0x1000000
-A cali-pri-k8s_ns.kube-system -m comment --comment "cali:d_ypsHpl3J96oOpx" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-pro-k8s_ns.default -m comment --comment "cali:DTsGE7pFaKbRuEBg" -j MARK --set-xmark 0x1000000/0x1000000
-A cali-pro-k8s_ns.default -m comment --comment "cali:4bIByWXruQ1DMcbo" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-pro-k8s_ns.kube-system -m comment --comment "cali:lDQGDZg5UANF5wIK" -j MARK --set-xmark 0x1000000/0x1000000
-A cali-pro-k8s_ns.kube-system -m comment --comment "cali:wn_dnW-P0COWnhhy" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-to-wl-dispatch -o cali45026c409f9 -m comment --comment "cali:c75T2Dgm3k-jJrbE" -g cali-tw-cali45026c409f9
-A cali-to-wl-dispatch -o calic0b238d4ce2 -m comment --comment "cali:qDV3G3z8-XF7ASpj" -g cali-tw-calic0b238d4ce2
-A cali-to-wl-dispatch -o caliec0efa8668a -m comment --comment "cali:0KGW9LSlkHoj3Pth" -g cali-tw-caliec0efa8668a
-A cali-to-wl-dispatch -m comment --comment "cali:jDu3duVnwTVndWys" -m comment --comment "Unknown interface" -j DROP
-A cali-tw-cali45026c409f9 -m comment --comment "cali:T8ds95eQAxnZl6cA" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A cali-tw-cali45026c409f9 -m comment --comment "cali:sBFjo942EoAZxbwi" -m conntrack --ctstate INVALID -j DROP
-A cali-tw-cali45026c409f9 -m comment --comment "cali:7mrDpuB_JSOiwD-w" -j MARK --set-xmark 0x0/0x1000000
-A cali-tw-cali45026c409f9 -m comment --comment "cali:SZ7jptebHBWtu0ut" -j cali-pri-k8s_ns.kube-system
-A cali-tw-cali45026c409f9 -m comment --comment "cali:XZUosCvhE-CFRBZf" -m comment --comment "Return if profile accepted" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-tw-cali45026c409f9 -m comment --comment "cali:UPdmXt0SUq5GpdCk" -m comment --comment "Drop if no profiles matched" -j DROP
-A cali-tw-calic0b238d4ce2 -m comment --comment "cali:k8kHsWO63lPZ_T5S" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A cali-tw-calic0b238d4ce2 -m comment --comment "cali:WcRO5jfNEyBl-P8e" -m conntrack --ctstate INVALID -j DROP
-A cali-tw-calic0b238d4ce2 -m comment --comment "cali:qgZ3s3ojXF7_0v41" -j MARK --set-xmark 0x0/0x1000000
-A cali-tw-calic0b238d4ce2 -m comment --comment "cali:l9FROf8cQyfmubvU" -j cali-pri-k8s_ns.default
-A cali-tw-calic0b238d4ce2 -m comment --comment "cali:i1mW8rmxu9TCd-T4" -m comment --comment "Return if profile accepted" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-tw-calic0b238d4ce2 -m comment --comment "cali:EOs-JJ221Us5p0EP" -m comment --comment "Drop if no profiles matched" -j DROP
-A cali-tw-caliec0efa8668a -m comment --comment "cali:_7y3hRmp6EU47Y0s" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A cali-tw-caliec0efa8668a -m comment --comment "cali:lqljOLOQn5ZkCC2p" -m conntrack --ctstate INVALID -j DROP
-A cali-tw-caliec0efa8668a -m comment --comment "cali:AGwqz_dfQJPaIJOa" -j MARK --set-xmark 0x0/0x1000000
-A cali-tw-caliec0efa8668a -m comment --comment "cali:IQNHtVteTcEbbzLF" -j cali-pri-k8s_ns.default
-A cali-tw-caliec0efa8668a -m comment --comment "cali:zFjCvYL15RsUfNaU" -m comment --comment "Return if profile accepted" -m mark --mark 0x1000000/0x1000000 -j RETURN
-A cali-tw-caliec0efa8668a -m comment --comment "cali:-GRpWsx8gV1ZNLvl" -m comment --comment "Drop if no profiles matched" -j DROP
-A cali-wl-to-host -m comment --comment "cali:Ee9Sbo10IpVujdIY" -j cali-from-wl-dispatch
-A cali-wl-to-host -m comment --comment "cali:nSZbcOoG1xPONxb8" -m comment --comment "Configured DefaultEndpointToHostAction" -j ACCEPT
COMMIT
# Completed on Wed Apr 25 21:25:23 2018
答案 0 :(得分：0)
这只是猜测，但我想我知道问题所在。
Kubernetes使用iptables管理广告连播和处理服务请求之间的流量，包括一些NAT规则。
当您在节点上调用服务时，您的请求也由iptables处理，其中包括基于源IP的NAT规则。
但是，看起来当您从同一节点调用服务时，您的数据包与服务的NAT规则不匹配，并且它们未得到正确处理。
Calico拥有NatOutgoing option，可以伪装成具有游泳池外目的地的所有数据包。
使用该选项，Calico将伪装包（用源节点的IP替换源IP），它将从节点本身作为包路由，并将被正确的服务的NAT规则捕获。
看起来可能会有帮助。
答案 1 :(得分：0)
我的机器的ip规则，它将阻止容器网络流量进入我的物理ip。删除ip规则后，问题就解决了。