Kubernetes POD超时等待卷附加/挂载

时间:2018-01-02 13:32:39

标签: kubernetes google-cloud-platform

我无法使用Kubernetes 1.8.0挂载GCE PersistentVolumes,每个POD都处于ContainerCreating状态。

此输出来自我为本实验所提供的测试环境。值得一提的是我使用的是Compute Engine,而不是Kubernetes Engine。

我没有配置任何云设置,如果这可能是根本原因,我会更加努力,但gcloud可以很好地完成工作,并且允许在此实验室环境中的所有虚拟机完全访问API。

工作人员的错误消息

Jan  2 13:03:58 worker-0 kubelet[1421]: E0102 13:03:58.733299    1421 kubelet.go:1628] Unable to mount volumes for pod "mysql-cgui-01-5c85f7dd86-gt2s8_default(ab17eaf2-efb6-11e7-a385-42010af0000a)": timeout expired waiting for volumes to attach/mount for pod "default"/"mysql-cgui-01-5c85f7dd86-gt2s8". list of unattached/unmounted volumes=[mysql-cgui-01]; skipping pod

POD说明

bofh:~$ kubectl describe pod mysql-cgui-01-5c85f7dd86-gt2s8
Name:           mysql-cgui-01-5c85f7dd86-gt2s8
Namespace:      default
Node:           worker-0/10.240.0.20
Start Time:     Tue, 02 Jan 2018 12:15:49 +0000
Labels:         name=mysql-cgui-01
                pod-template-hash=1741938842
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"default","name":"mysql-cgui-01-5c85f7dd86","uid":"ab10f9ef-efb6-11e7-a385-42010af...
Status:         Pending
IP:
Created By:     ReplicaSet/mysql-cgui-01-5c85f7dd86
Controlled By:  ReplicaSet/mysql-cgui-01-5c85f7dd86
Containers:
  mysql-cgui-01:
    Container ID:
    Image:          external/mysql:latest
    Image ID:
    Port:           3306/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Mounts:
      /data/mysql from mysql-cgui-01 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-tb6sm (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  mysql-cgui-01:
    Type:       GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine)
    PDName:     mysql-cgui-01
    FSType:     ext4
    Partition:  0
    ReadOnly:   false
  default-token-tb6sm:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-tb6sm
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     <none>
Events:
  Type     Reason                 Age                From               Message
  ----     ------                 ----               ----               -------
  Normal   Scheduled              55m                default-scheduler  Successfully assigned mysql-cgui-01-5c85f7dd86-gt2s8 to worker-0
  Normal   SuccessfulMountVolume  55m                kubelet, worker-0  MountVolume.SetUp succeeded for volume "default-token-tb6sm"
  Warning  FailedMount            41m (x6 over 53m)  kubelet, worker-0  Unable to mount volumes for pod "mysql-cgui-01-5c85f7dd86-gt2s8_default(ab17eaf2-efb6-11e7-a385-42010af0000a)": timeout expired waiting for volumes to attach/mount for pod "default"/"mysql-cgui-01-5c85f7dd86-gt2s8". list of unattached/unmounted volumes=[mysql-cgui-01]
  Warning  FailedSync             41m (x6 over 53m)  kubelet, worker-0  Error syncing pod
  Normal   SuccessfulMountVolume  38m                kubelet, worker-0  MountVolume.SetUp succeeded for volume "default-token-tb6sm"
  Warning  FailedMount            4m (x15 over 36m)  kubelet, worker-0  Unable to mount volumes for pod "mysql-cgui-01-5c85f7dd86-gt2s8_default(ab17eaf2-efb6-11e7-a385-42010af0000a)": timeout expired waiting for volumes to attach/mount for pod "default"/"mysql-cgui-01-5c85f7dd86-gt2s8". list of unattached/unmounted volumes=[mysql-cgui-01]
  Warning  FailedSync             2m (x16 over 36m)  kubelet, worker-0  Error syncing pod

使用worker-0

中的gcloud进行测试
worker-0:~$ gcloud compute disks list
NAME           ZONE            SIZE_GB  TYPE         STATUS
bofh           europe-west1-d  20       pd-standard  READY
controller-0   europe-west1-c  200      pd-standard  READY
controller-1   europe-west1-c  200      pd-standard  READY
controller-2   europe-west1-c  200      pd-standard  READY
mysql-cgui-01  europe-west1-c  10       pd-standard  READY
mysql-cgui-02  europe-west1-c  10       pd-standard  READY
worker-0       europe-west1-c  200      pd-standard  READY
worker-1       europe-west1-c  200      pd-standard  READY
worker-2       europe-west1-c  200      pd-standard  READY

Worker-0 kubelet标志

ExecStart=/usr/local/bin/kubelet \
  --allow-privileged=true \
  --anonymous-auth=false \
  --authorization-mode=Webhook \
  --client-ca-file=/var/lib/kubernetes/ca.pem \
  --cluster-dns=10.32.0.10 \
  --cluster-domain=cluster.local \
  --container-runtime=docker \
  --image-pull-progress-deadline=2m \
  --kubeconfig=/var/lib/kubelet/kubeconfig \
  --network-plugin=cni \
  --pod-cidr=10.200.0.0/24 \
  --register-node=true \
  --require-kubeconfig \
  --runtime-request-timeout=15m \
  --tls-cert-file=/var/lib/kubelet/worker-0.pem \
  --tls-private-key-file=/var/lib/kubelet/worker-0-key.pem \
  --cloud-provider=gce \
  --v=2

Worker-0 kube-proxy flags

ExecStart=/usr/local/bin/kube-proxy \
  --cluster-cidr=10.200.0.0/16 \
  --kubeconfig=/var/lib/kube-proxy/kubeconfig \
  --proxy-mode=iptables \
  --v=2

控制器kube-scheduler标志

ExecStart=/usr/local/bin/kube-scheduler \
  --leader-elect=true \
  --master=http://127.0.0.1:8080 \
  --v=2

Controller kube-controllermanager flags

ExecStart=/usr/local/bin/kube-controller-manager \
  --address=0.0.0.0 \
  --cluster-cidr=10.200.0.0/16 \
  --cluster-name=kubernetes \
  --cluster-signing-cert-file=/var/lib/kubernetes/ca.pem \
  --cluster-signing-key-file=/var/lib/kubernetes/ca-key.pem \
  --leader-elect=true \
  --master=http://127.0.0.1:8080 \
  --root-ca-file=/var/lib/kubernetes/ca.pem \
  --service-account-private-key-file=/var/lib/kubernetes/ca-key.pem \
  --service-cluster-ip-range=10.32.0.0/24 \
  --cloud-provider=gce \
  --allocate-node-cidrs=false \
  --configure-cloud-routes=false \
  --v=2

controller kube-apiserver flags

ExecStart=/usr/local/bin/kube-apiserver \
  --admission-control=Initializers,NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \
  --advertise-address=10.240.0.10 \
  --allow-privileged=true \
  --apiserver-count=3 \
  --audit-log-maxage=30 \
  --audit-log-maxbackup=3 \
  --audit-log-maxsize=100 \
  --audit-log-path=/var/log/audit.log \
  --authorization-mode=Node,RBAC \
  --bind-address=0.0.0.0 \
  --client-ca-file=/var/lib/kubernetes/ca.pem \
  --enable-swagger-ui=true \
  --etcd-cafile=/var/lib/kubernetes/ca.pem \
  --etcd-certfile=/var/lib/kubernetes/kubernetes.pem \
  --etcd-keyfile=/var/lib/kubernetes/kubernetes-key.pem \
  --etcd-servers=https://10.240.0.10:2379,https://10.240.0.11:2379,https://10.240.0.12:2379 \
  --event-ttl=1h \
  --experimental-encryption-provider-config=/var/lib/kubernetes/encryption-config.yaml \
  --insecure-bind-address=127.0.0.1 \
  --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \
  --kubelet-client-certificate=/var/lib/kubernetes/kubernetes.pem \
  --kubelet-client-key=/var/lib/kubernetes/kubernetes-key.pem \
  --kubelet-https=true \
  --runtime-config=api/all \
  --service-account-key-file=/var/lib/kubernetes/ca-key.pem \
  --service-cluster-ip-range=10.32.0.0/24 \
  --service-node-port-range=30000-32767 \
  --tls-ca-file=/var/lib/kubernetes/ca.pem \
  --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \
  --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \
  --cloud-provider=gce \
  --v=2

版本

bofh:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"6e937839ac04a38cac63e6a7a306c5d035fe7b0a", GitTreeState:"clean", BuildDate:"2017-09-28T22:57:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"6e937839ac04a38cac63e6a7a306c5d035fe7b0a", GitTreeState:"clean", BuildDate:"2017-09-28T22:46:41Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
bofh:~$ kubectl get nodes
NAME       STATUS    ROLES     AGE       VERSION
worker-0   Ready     <none>    12d       v1.8.0
worker-1   Ready     <none>    12d       v1.8.0
worker-2   Ready     <none>    12d       v1.8.0

PV和PVC信息

bofh:~$ kubectl get pv
No resources found.
bofh:~$ kubectl get pvc
No resources found.
bofh:~$ kubectl describe pv
bofh:~$ kubectl describe pvc

StorageClass config

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard

存储类输出

bofh:~$ kubectl get sc
NAME       PROVISIONER
slow       kubernetes.io/gce-pd
standard   kubernetes.io/gce-pd

2 个答案:

答案 0 :(得分:1)

您需要将云提供商标志添加到apiserver,kubelet和controller-manager。

然后,您还需要为GCE配置存储类

答案 1 :(得分:0)

我们注意到一个类似的问题,并认为它可能与此标志有关:

--experimental-encryption-provider-config=/var/lib/kubernetes/encryption-config.yaml

尝试删除它并查看错误是否停止。