kube-api-server保持上下

时间:2018-07-01 19:44:36

标签: kubernetes

我们有一个HA Kubernetes集群(3个主节点+ N个节点)。直到昨天一切都运行良好,我发现我无法访问所有主节点,所有kube-api ports(6443)都无法正常工作。深入研究之后,我发现kube-api-server一直在Click here for image上下波动,我不知道发生了什么,这是该破碎的kube-api映像的错误日志 Click here

错误日志

I0701 19:36:33.809836       1 trace.go:76] Trace[815193926]: "Patch /api/v1/namespaces/kube-system/events/kube-flannel-ds-gptzq.153d39dac9607753" (started: 2018-07-01 19:36:03.809010257 +0000 UTC m=+33.245566917) (total time: 30.000775211s):
Trace[815193926]: [30.000775211s] [30.000715052s] END
I0701 19:36:34.915664       1 trace.go:76] Trace[1130857390]: "Patch /api/v1/namespaces/kube-system/events/kube-flannel-ds-fr8ck.153d2e95d8a6ae14" (started: 2018-07-01 19:36:04.914928761 +0000 UTC m=+34.351485427) (total time: 30.000694205s):
Trace[1130857390]: [30.000694205s] [30.000620863s] END
E0701 19:36:37.482204       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.Pod: the server was unable to return a response in the time allotted, but may still be processing the request (get pods)
E0701 19:36:37.489899       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *rbac.Role: the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io)
E0701 19:36:37.489933       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.PersistentVolume: the server was unable to return a response in the time allotted, but may still be processing the request (get persistentvolumes)
E0701 19:36:37.491074       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1.Service: the server was unable to return a response in the time allotted, but may still be processing the request (get services)
E0701 19:36:37.492678       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.LimitRange: the server was unable to return a response in the time allotted, but may still be processing the request (get limitranges)
E0701 19:36:37.492880       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/client/informers/internalversion/factory.go:74: Failed to list *apiregistration.APIService: the server was unable to return a response in the time allotted, but may still be processing the request (get apiservices.apiregistration.k8s.io)
E0701 19:36:37.492963       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/apiextensions-apiserver/pkg/client/informers/internalversion/factory.go:74: Failed to list *apiextensions.CustomResourceDefinition: the server was unable to return a response in the time allotted, but may still be processing the request (get customresourcedefinitions.apiextensions.k8s.io)
E0701 19:36:37.492999       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.ResourceQuota: the server was unable to return a response in the time allotted, but may still be processing the request (get resourcequotas)
E0701 19:36:37.497459       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.ServiceAccount: the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts)
E0701 19:36:37.497748       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *core.Secret: the server was unable to return a response in the time allotted, but may still be processing the request (get secrets)
E0701 19:36:37.499772       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1.Namespace: the server was unable to return a response in the time allotted, but may still be processing the request (get namespaces)
E0701 19:36:37.499959       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *rbac.ClusterRoleBinding: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io)
E0701 19:36:37.500661       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:87: Failed to list *v1beta1.VolumeAttachment: the server was unable to return a response in the time allotted, but may still be processing the request (get volumeattachments.storage.k8s.io)
E0701 19:36:37.500778       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *storage.StorageClass: the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io)
E0701 19:36:37.500893       1 reflector.go:205] k8s.io/kubernetes/pkg/client/informers/informers_generated/internalversion/factory.go:86: Failed to list *rbac.ClusterRole: the server was unable to return a response in the time allotted, but may still be processing the request (get clusterroles.rbac.authorization.k8s.io)
E0701 19:36:37.501017       1 authentication.go:63] Unable to authenticate ...

0 个答案:

没有答案