kubeadm init卡住了代理和禁止的错误

时间:2018-02-22 18:07:16

标签: proxy kubernetes init kubeadm

我花了两天时间尝试在虚拟机上初始化kubernetes的主节点。

我不知道问题是什么,我得到一些"禁止"并且"连接被拒绝"错误

我还设置了我的http_proxy env var,如下所示:

$ export http_proxy=http://proxy-ip:port/
$ export https_proxy=http://proxy-ip:port/

我的kubeadm版本:

kubeadm version: &version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-02-07T11:55:20Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

每当我做的时候:

kubeadm init --pod-network-cidr=10.244.0.0/16

我得到了这个结果:

[init] Using Kubernetes version: v1.9.3
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
    [WARNING FileExisting-crictl]: crictl not found in system path
    [WARNING HTTPProxy]: Connection to "https://IP:6443" uses proxy "http://proxy-ip:port". If that is not intended, adjust your proxy settings
    [WARNING HTTPProxyCIDR]: connection to "10.96.0.0/12" uses proxy "http://proxy-ip:port". This may lead to malfunctional cluster setup. Make sure that Pod and Services IP ranges specified correctly as exceptions in proxy configuration
    [WARNING HTTPProxyCIDR]: connection to "10.244.0.0/16" uses proxy "http://proxy-ip:port". This may lead to malfunctional cluster setup. Make sure that Pod and Services IP ranges specified correctly as exceptions in proxy configuration
[preflight] Starting the kubelet service
[certificates] Generated ca certificate and key.
[certificates] Generated apiserver certificate and key.
[certificates] apiserver serving cert is signed for DNS names [mydns kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 IP]
[certificates] Generated apiserver-kubelet-client certificate and key.
[certificates] Generated sa key and public key.
[certificates] Generated front-proxy-ca certificate and key.
[certificates] Generated front-proxy-client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml"
[init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests".
[init] This might take a minute or longer if the control plane images have to be pulled.

......它被困在那里。这是我的kubelet日志

Feb 22 11:17:30   kubelet: E0222 11:17:30.486017    3983 summary.go:92] Failed to get system container stats for "/system.slice/kubelet.service": failed to get cgroup stats for "/system.slice/kubelet.service": failed to get container info for "/system.slice/kubelet.service": unknown container "/system.slice/kubelet.service"
Feb 22 11:17:30   kubelet: E0222 11:17:30.486054    3983 summary.go:92] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"
Feb 22 11:17:30   kubelet: W0222 11:17:30.491931    3983 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Feb 22 11:17:30   kubelet: E0222 11:17:30.492066    3983 kubelet.go:2104] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Feb 22 11:17:33   journal: E0222 16:17:33.272564       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: Get https://IP:6443/api/v1/nodes?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.273579       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: Get https://IP:6443/api/v1/persistentvolumes?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.273635       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: Get https://IP:6443/apis/apps/v1beta1/statefulsets?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.273689       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: Get https://IP:6443/api/v1/persistentvolumeclaims?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.275024       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: Get https://IP:6443/api/v1/replicationcontrollers?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.275173       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: Get https://IP:6443/api/v1/services?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.275200       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: Get https://IP:6443/apis/policy/v1beta1/poddisruptionbudgets?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.275314       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: Get https://IP:6443/api/v1/pods?fieldSelector=spec.schedulerName%3Ddefault-scheduler%2Cstatus.phase%21%3DFailed%2Cstatus.phase%21%3DSucceeded&limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:33   journal: E0222 16:17:33.275318       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: Get https://IP:6443/apis/extensions/v1beta1/replicasets?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:35   kubelet: W0222 11:17:35.492831    3983 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Feb 22 11:17:35   kubelet: E0222 11:17:35.493249    3983 kubelet.go:2104] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Feb 22 11:17:36   journal: E0222 16:17:36.189616       1 leaderelection.go:224] error retrieving resource lock kube-system/kube-controller-manager: Get https://IP:6443/api/v1/namespaces/kube-system/endpoints/kube-controller-manager: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.278107       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolume: Get https://IP:6443/api/v1/persistentvolumes?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.278220       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: Get https://IP:6443/api/v1/nodes?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.278320       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.ReplicaSet: Get https://IP:6443/apis/extensions/v1beta1/replicasets?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.278556       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.StatefulSet: Get https://IP:6443/apis/apps/v1beta1/statefulsets?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.278665       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.PersistentVolumeClaim: Get https://IP:6443/api/v1/persistentvolumeclaims?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.278953       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.ReplicationController: Get https://IP:6443/api/v1/replicationcontrollers?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.279453       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Service: Get https://IP:6443/api/v1/services?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.279748       1 reflector.go:205] k8s.io/kubernetes/plugin/cmd/kube-scheduler/app/server.go:590: Failed to list *v1.Pod: Get https://IP:6443/api/v1/pods?fieldSelector=spec.schedulerName%3Ddefault-scheduler%2Cstatus.phase%21%3DFailed%2Cstatus.phase%21%3DSucceeded&limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:39   journal: E0222 16:17:39.282080       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1beta1.PodDisruptionBudget: Get https://IP:6443/apis/policy/v1beta1/poddisruptionbudgets?limit=500&resourceVersion=0: Forbidden
Feb 22 11:17:40   kubelet: W0222 11:17:40.493931    3983 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Feb 22 11:17:40   kubelet: E0222 11:17:40.494041    3983 kubelet.go:2104] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

每当我看到我的docker ps时,我都会看到api-server上升然后关闭。

3 个答案:

答案 0 :(得分:1)

无法获取系统容器统计信息:

Feb 22 11:17:30   kubelet: E0222 11:17:30.486017    3983 summary.go:92] Failed to get system container stats for "/system.slice/kubelet.service": failed to get cgroup stats for "/system.slice/kubelet.service": failed to get container info for "/system.slice/kubelet.service": unknown container "/system.slice/kubelet.service"
Feb 22 11:17:30   kubelet: E0222 11:17:30.486054    3983 summary.go:92] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"

确保kubelet和docker的cgroup相同。

docker info | grep -i cgroup
cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

如果不同,则更新/etc/systemd/system/kubelet.service.d/10-kubeadm.conf以将docker的cgroup设置为kubelet。在文件中查找 cgroup-driver 。需要重新加载/重新启动kubelet systemd服务。

请参阅Troubleshooting kubeadm并了解所有有用的内容。

无法列出* v1.Node:获取https://IP:6443/api/v1/nodes

Feb 22 11:17:30   kubelet: W0222 11:17:30.491931    3983 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Feb 22 11:17:30   kubelet: E0222 11:17:30.492066    3983 kubelet.go:2104] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Feb 22 11:17:33   journal: E0222 16:17:33.272564       1 reflector.go:205] k8s.io/kubernetes/vendor/k8s.io/client-go/informers/factory.go:86: Failed to list *v1.Node: Get https://IP:6443/api/v1/nodes?limit=500&resourceVersion=0: Forbidden

请确保已设置正确的群集API服务器IP。

https://**IP**:6443/api/v1/nodes <----- "**IP"** should be the API server IP address?

明确指定:

kubeadm init --apiserver-advertise-address=<The IP of the API server>

请参阅kubeadm init - optionsInitializing your master

kubeadm reset

或许需要重新安装群集。 kubeadm reset不会让你重新运行kubeadm init并按预期重新创建集群。需要手动清理覆盖网络文件等。

请参阅"Failed to setup network for pod \ using network plugins \"cni\": no IP addresses available in network: podnet; Skipping pod"

答案 1 :(得分:1)

如果kubeadm存在使用错误代理的问题,请不要尝试使用

修改环境变量

导出http_proxy = .....

因为kubeadm不考虑这些变量。 kubeadm查看/ etc / environment以找到要使用的代理。因此,请在/ etc / environment

中声明您的代理

(根据我的经验)

答案 2 :(得分:0)

我用no_proxy env变量修复它。