在k8s centos7裸机集群上重新创建部署/吊舱后,服务将不允许流量到吊舱:无主机路由

时间:2020-05-03 13:20:20

标签: kubernetes

创建了一个裸机多主群集1.18版本:

NAME            STATUS   ROLES    AGE   VERSION
master1   Ready    master   9d    v1.18.0
master2   Ready    master   9d    v1.18.0
master3   Ready    master   9d    v1.18.0
worker1   Ready    <none>   9d    v1.18.0
worker2   Ready    <none>   9d    v1.18.0
worker3   Ready    <none>   9d    v1.18.0

kubectl create deploy la --image = linuxacademycontent / kubeserve:v1
kubectl规模部署la --replica = 6
kubectl取得广告连播

curlpod                1/1     Running   0          55m
la-65776bc88f-4kp95    1/1     Running   0          42m
la-65776bc88f-7lb9f    1/1     Running   0          42m
la-65776bc88f-dgzsz    1/1     Running   0          42m
la-65776bc88f-l2hs2    1/1     Running   0          43m
la-65776bc88f-vkcts    1/1     Running   0          42m
la-65776bc88f-wkm2k    1/1     Running   0          42m

kubectl公开部署la --port = 80 --target-port = 80
kubectl获取svc

kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP   75m
la           ClusterIP   10.103.199.187   <none>        80/TCP    46m

kubectl get ep

kubernetes   172.31.16.67:6443                                       78m
la           10.244.1.6:80,10.244.1.7:80,10.244.2.5:80 + 3 more...   49m

我可以通过从testpod中卷曲来验证所有功能是否正常-curlpod
kubectl exec -it curlpod bash

This is v1 pod la-65776bc88f-dgzsz



问题是,如果我现在删除部署(或删除所有吊舱),然后重新创建它们,则该服务将不允许流量进入新吊舱
这是重现问题的方法。 kubectl删除部署la

curlpod                1/1     Running       0          71m
la-65776bc88f-4kp95    1/1     Terminating   0          58m
la-65776bc88f-7lb9f    1/1     Terminating   0          58m
la-65776bc88f-dgzsz    1/1     Terminating   0          58m
la-65776bc88f-l2hs2    1/1     Terminating   0          59m
la-65776bc88f-vkcts    1/1     Terminating   0          58m
la-65776bc88f-wkm2k    1/1     Terminating   0          58m

kubectl创建部署la --image = linuxacademycontent / kubeserve:v1
kubectl规模部署la --replicas = 6
kubectl吊舱

NAME                   READY   STATUS    RESTARTS   AGE
curlpod                1/1     Running   0          73m
la-65776bc88f-5q7d8    1/1     Running   0          6s
la-65776bc88f-b69z2    1/1     Running   0          10s
la-65776bc88f-j2t4t    1/1     Running   0          6s
la-65776bc88f-ndxs5    1/1     Running   0          6s
la-65776bc88f-ns2gt    1/1     Running   0          6s
la-65776bc88f-vjp8x    1/1     Running   0          6s

一切看起来不错。但是,该服务不允许对新Pod进行任何流量
kubectl exec -it curlpod bash

root@curlpod:/app# curl la
curl: (7) Failed to connect to la port 80: No route to host
root@curlpod:/app# curl 10.103.199.187   #name resolution seems fine
curl: (7) Failed to connect to la port 80: No route to host

我仍然可以单独到达豆荚。例如la-65776bc88f-5q7d8的IP:10.244.5.13

root@test-pod:/app# curl 10.244.5.13
This is v1 pod la-65776bc88f-5q7d8

因此,我认为是服务无法路由流量,但是我可以看到服务没有变化,所有新的Pod均被列为服务的端点。 kubectl get svc

NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP   93m
la           ClusterIP   10.103.199.187   <none>        80/TCP    64m

kubectl get ep

NAME         ENDPOINTS                                               AGE
kubernetes   172.31.16.67:6443                                       93m
la           10.244.5.13,10.244.1.9:80,10.244.2.7:80 + 3 more...   64m

某处阻止了从服务到新Pod的流量。

现在,如果我们重新启动svc,问题将消失!
kubectl删除svc la
服务“ la”已删除

kubectl公开部署la --port = 80 --target-port = 80
服务/已暴露

kubectl get svc

kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   108m
la           ClusterIP   10.96.124.200   <none>        80/TCP    71s

kubectl exec -it curlpod bash

This is v1 pod la-65776bc88f-z24qn

尝试了一些替代方法,例如使用yaml创建部署,服务等相同的问题。 还尝试使用kubectl缩放来缩放。但是,我在部署中创建了新的Pod,从服务到Pod的流量将始终被阻止。

欢迎任何帮助或建议!

添加法兰绒日志

kubectl logs --namespace kube-system kube-flannel-ds-amd64-tbglf
I0423 15:15:52.311421       1 main.go:518] Determining IP address of default interface
I0423 15:15:52.312644       1 main.go:531] Using interface with name eth0 and address 10.30.193.13
I0423 15:15:52.312669       1 main.go:548] Defaulting external address to interface address (10.30.193.13)
W0423 15:15:52.312689       1 client_config.go:517] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0423 15:15:52.323517       1 kube.go:119] Waiting 10m0s for node controller to sync
I0423 15:15:52.324491       1 kube.go:306] Starting kube subnet manager
I0423 15:15:53.323919       1 kube.go:126] Node controller sync successful
I0423 15:15:53.323976       1 main.go:246] Created subnet manager: Kubernetes Subnet Manager - uk02sybk8smp3
I0423 15:15:53.323983       1 main.go:249] Installing signal handlers
I0423 15:15:53.324084       1 main.go:390] Found network config - Backend type: vxlan
I0423 15:15:53.324176       1 vxlan.go:121] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
I0423 15:15:53.365832       1 main.go:355] Current network or subnet (10.244.0.0/16, 10.244.2.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules
I0423 15:15:53.367815       1 iptables.go:167] Deleting iptables rule: -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN
I0423 15:15:53.409089       1 iptables.go:167] Deleting iptables rule: -s 0.0.0.0/0 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I0423 15:15:53.410118       1 iptables.go:167] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN
I0423 15:15:53.411017       1 iptables.go:167] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j MASQUERADE --random-fully
I0423 15:15:53.412076       1 main.go:305] Setting up masking rules
I0423 15:15:53.412916       1 main.go:313] Changing default FORWARD chain policy to ACCEPT
I0423 15:15:53.412999       1 main.go:321] Wrote subnet file to /run/flannel/subnet.env
I0423 15:15:53.413009       1 main.go:325] Running backend.
I0423 15:15:53.413017       1 main.go:343] Waiting for all goroutines to exit
I0423 15:15:53.413038       1 vxlan_network.go:60] watching for new subnet leases
I0423 15:15:53.510136       1 iptables.go:145] Some iptables rules are missing; deleting and recreating rules
I0423 15:15:53.510158       1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
I0423 15:15:53.510293       1 iptables.go:145] Some iptables rules are missing; deleting and recreating rules
I0423 15:15:53.510301       1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 -j ACCEPT
I0423 15:15:53.511096       1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I0423 15:15:53.512091       1 iptables.go:167] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.2.0/24 -j RETURN
I0423 15:15:53.512206       1 iptables.go:167] Deleting iptables rule: -d 10.244.0.0/16 -j ACCEPT
I0423 15:15:53.512998       1 iptables.go:167] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully
I0423 15:15:53.513970       1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
I0423 15:15:53.609595       1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 -j ACCEPT
I0423 15:15:53.612312       1 iptables.go:155] Adding iptables rule: -d 10.244.0.0/16 -j ACCEPT
I0423 15:15:53.613132       1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I0423 15:15:53.710527       1 iptables.go:155] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.2.0/24 -j RETURN
I0423 15:15:53.712938       1 iptables.go:155] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully

kube-proxy充满

E0504 08:25:36.251542       1 proxier.go:1533] Failed to sync endpoint for service: 10.30.218.46:30080/TCP, err: parseIP Error ip=[10 244 5 3 0 0 0 0 0 0 0 0 0 0 0 0]
E0504 08:25:36.251632       1 proxier.go:1950] Failed to list IPVS destinations, error: parseIP Error ip=[10 244 5 3 0 0 0 0 0 0 0 0 0 0 0 0]

0 个答案:

没有答案
相关问题