Question

我正在尝试在kubernetes v1.9.2中测试local persistent volume。

从我收集的内容（我可能错了！）我无法使用kubeadm添加这些功能门：

$ sudo kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.2", GitCommit:"5fa2db2bd46ac79e5e00a4e6ed24191080aa463b", GitTreeState:"clean", BuildDate:"2018-01-18T09:42:01Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

$ kubeadm init --help
...
      --feature-gates string                    A set of key=value pairs that describe feature gates for various features. Options are:
        CoreDNS=true|false (ALPHA - default=false)
        DynamicKubeletConfig=true|false (ALPHA - default=false)
        SelfHosting=true|false (ALPHA - default=false)
        StoreCertsInSecrets=true|false (ALPHA - default=false)
...

sooo ...我做一个正常的kubeadm初始化，然后继续攻击：

/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

带

Environment="KUBELET_FEATURE_GATES_ARGS=--feature-gates=PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CGROUP_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS $KUBELET_FEATURE_GATES_ARGS

并重新加载/重新启动kubelet。

好吧......试试创建pv：

$ cat local-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-local-pv
  annotations:
    "volume.alpha.kubernetes.io/node-affinity": '{
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          { "matchExpressions": [
            { "key": "kubernetes.io/hostname",
              "operator": "In",
              "values": ["dhcp-nebula-129-230"]
            }
          ]}
         ]}
        }'
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /mnt/disks/fs2

$ kubectl create -f local-pv.yaml
The PersistentVolume "example-local-pv" is invalid:
* metadata.annotations: Forbidden: Storage node affinity is disabled by

特征栅 * spec.local：Forbidden：功能门

禁用本地卷

AHA！我说......我也要改变kube-apiserver！

所以我编辑/etc/kubernetes/manifests/kube-apiserver.yaml并将以下内容附加到Command：

--feature-gates=PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true

并且当kubectl死亡时，api服务器死了，我没有kubeapi-server。 :(

帮助？

Answer 1

您需要在kubeadm配置中在api，scheduler和controller上设置功能门：

apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
apiServerExtraArgs:
  service-node-port-range: 80-32767
  feature-gates: "PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"
controllerManagerExtraArgs:
  feature-gates: "PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"
schedulerExtraArgs:
  feature-gates: "PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"

存储类示例：

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mysql-data
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

PVC示例：

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app: mariadb
  name: mysql-mariadb-0
  namespace: mysql
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: mysql-data
  selector:
    matchLabels:
      pod-name: mariadb-0

PV示例：

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-mariadb-0
  labels:
    pod-name: mariadb-0
  annotations:
    "volume.alpha.kubernetes.io/node-affinity": '{
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          { "matchExpressions": [
              { "key": "kubernetes.io/hostname",
                "operator": "In",
                "values": ["prod-mysql-0"]
              }
          ]}
        ]}}'
spec:
  capacity:
    storage: 2Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: mysql-data
  local:
    path: /mnt/local-storage/mysql-data-0

Answer 2

编辑/etc/kubernetes/manifests/kube-apiserver.yaml并删除kube-apiserver窗格以重新启动它。通过编辑确认您的更改在命令字段下

 kubectl -n kube-system edit pod kube-apiserver**

spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=192.168.0.30
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379
    - --insecure-port=0
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --requestheader-allowed-names=front-proxy-client
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --requestheader-group-headers=X-Remote-Group
    - --requestheader-username-headers=X-Remote-User
    - --secure-port=6443
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --service-cluster-ip-range=10.96.0.0/12
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --feature-gates=VolumeSnapshotDataSource=true
    image: k8s.gcr.io/kube-apiserver:v1.17.3

Answer 3

对于v1.15中的Kubernetes，请执行以下操作：

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.15.0
apiServer:
  extraArgs:
    advertise-address: 192.168.1.216
    feature-gates: SCTPSupport=true

kubeadm / kubectl / kube-apiserver打开要素大门

3 个答案: