具有EBS的Kubernetes永久卷(在EC2实例中)

时间:2020-05-23 13:17:37

标签: mysql amazon-ec2 kubernetes amazon-ebs persistent-volumes

当前,我正在尝试使用带有Kubernetes和Kubeadm的MYSQL数据库进行卷持久化。 该环境基于使用EBS存储磁盘的Amazon EC2实例。

正如您在存储类下面看到的那样,为了实现mysql持久性,已实现了持久卷以及持久卷声明。 但是,当我尝试部署mysql pod(在附加的映像上)时会发生错误。

mysql-pv.yml:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
reclaimPolicy: Retain
mountOptions:
  - debug
volumeBindingMode: Immediate
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-pv
  labels:
    type: amazonEBS
spec:
  capacity:
    storage: 5Gi
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  awsElasticBlockStore:
    volumeID: vol-ID
    fsType: ext4
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

mysql.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - image: mysql:5.7.30
          name: mysql
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: MYPASSWORD
          ports:
            - containerPort: 3306
              name: mysql
          volumeMounts:
            - name: mysql-persistent-storage
              mountPath: /var/lib/mysql
      volumes:
        - name: mysql-persistent-storage
          persistentVolumeClaim:
            claimName: mysql-pv-claim
---
apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  type: NodePort
  ports:
    - port: 3306
      targetPort: 3306
      nodePort: 31306
  selector:
    app: mysql

这是我的mysql pod描述:

Name:           mysql-5c9788fc65-jq2nh
Namespace:      default
Priority:       0
Node:           ip-172-31-31-210/172.31.31.210
Start Time:     Sat, 23 May 2020 12:19:24 +0000
Labels:         app=mysql
                pod-template-hash=5c9788fc65
Annotations:    <none>
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  ReplicaSet/mysql-5c9788fc65
Containers:
  mysql:
    Container ID:   
    Image:          mysql:5.7.30
    Image ID:       
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:
      MYSQL_ROOT_PASSWORD:  MYPASS
    Mounts:
      /data/ from mysql-persistent-storage (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-cshk2 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  mysql-persistent-storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  mysql-pv-claim
    ReadOnly:   false
  default-token-cshk2:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-cshk2
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s

这是我得到的错误:

Events:
Type     Reason       Age        From                       Message
----     ------       ----       ----                       -------
Normal   Scheduled    <unknown>  default-scheduler          
Successfully assigned default/mysql-5c9788fc65-jq2nh to ip-172-31-31-210
Warning  FailedMount  39m        kubelet, ip-172-31-31-210  MountVolume.SetUp failed for volume "mysql-pv" : mount failed: exit status 32

Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/29d5cee7-da11-4a0c-b5aa-e262f919d1ba/volumes/kubernetes.io~aws-ebs/mysql-pv --scope -- mount  -o bind /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-06212746d87534157 /var/lib/kubelet/pods/29d5cee7-da11-4a0c-b5aa-e262f919d1ba/volumes/kubernetes.io~aws-ebs/mysql-pv
Output: Running scope as unit: run-r11fefbbda1d241c2985931d3adaaa969.scope
mount: /var/lib/kubelet/pods/29d5cee7-da11-4a0c-b5aa-e262f919d1ba/volumes/kubernetes.io~aws-ebs/mysql-pv: special device /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-06212746d87534157 does not exist.
Warning  FailedMount  39m  kubelet, ip-172-31-31-210  MountVolume.SetUp failed for volume "mysql-pv" : mount failed: exit status 32

有人可以帮助我吗?

2 个答案:

答案 0 :(得分:0)

检查PVC和PVC是否处于绑定状态。

kubectl describe pvc mysql-pv-claim kubectl describe pv mysql-pv 您是否安装了EBS CSI driver

其他原因可能是,我认为您错过了添加-cloud-provider = aws 的选项,这是CCM要求的节点。查看类似的issue

以下链接具有所有IAM权限以及一个有效的示例,该示例说明如何根据cluster configuration上的文档在Kubernetes中创建和安装EBS卷以使用EBS。

使用kubeadm配置时,配置定义在: /var/lib/kubelet/config.yaml /var/lib/kubelet/kubeadm-flags.env

如果集群是使用 kubeadm 部署的,则在kubeadm-flags.env中的所有节点上定义环境变量 要手动解决此问题,您需要将--cloud-provider=aws标签添加到kubeadm-flags.env并重新启动服务,这将解决此问题:

systemctl daemon-reload && systemctl restart kubelet 或为kubeadm提供以下configuration。根据您的情况将openstack更改为AWS。

请检查以下blog,以获得更好的理解。

kind: InitConfiguration
nodeRegistration:
  kubeletExtraArgs:
    cloud-provider: "aws"
    cloud-config: "/etc/kubernetes/cloud.conf"
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
  extraArgs:
    cloud-provider: "aws"
    #cloud-config: "/etc/kubernetes/cloud.conf"
  extraVolumes:
  - name: cloud
    hostPath: "/etc/kubernetes/cloud.conf"
    mountPath: "/etc/kubernetes/cloud.conf"
controllerManager:
  extraArgs:
    cloud-provider: "aws"
    #cloud-config: "/etc/kubernetes/cloud.conf"
  extraVolumes:
  - name: cloud
    hostPath: "/etc/kubernetes/cloud.conf"
    mountPath: "/etc/kubernetes/cloud.conf"```

答案 1 :(得分:0)

以下是一些其他信息:

kubectl描述pvc mysql-pv-claim

Name:          mysql-pv-claim
Namespace:     default
StorageClass:  standard
Status:        Bound 
Volume:        mysql-pv
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
           pv.kubernetes.io/bound-by-controller: yes
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      5Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Mounted By:    mysql-5c9788fc65-jq2nh
Events:        <none>

kubectl描述pv mysql-pv

Name:            mysql-pv
Labels:          type=amazonEBS
Annotations:     pv.kubernetes.io/bound-by-controller: yes
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    standard
Status:          Bound
Claim:           default/mysql-pv-claim
Reclaim Policy:  Retain
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        5Gi
Node Affinity:   <none>
Message:         
Source:
    Type:       AWSElasticBlockStore (a Persistent Disk resource in AWS)
    VolumeID:   vol-06212746d87534157
    FSType:     ext4
    Partition:  0
    ReadOnly:   false
Events:         <none>

lsblk

NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0         7:0    0   18M  1 loop /snap/amazon-ssm-agent/1566
loop1         7:1    0 93.9M  1 loop /snap/core/9066
loop2         7:2    0 93.8M  1 loop /snap/core/8935
nvme0n1     259:0    0   10G  0 disk 
nvme1n1     259:1    0   15G  0 disk 
└─nvme1n1p1 259:2    0   15G  0 part /

我想使用nvme0n1。

我没有kubelet和kube-controller-manager.log文件日志。