Question

我正在尝试使用kubeadm作为安装程序并将F5作为负载均衡器（不能使用HAproxy）来设置Kubernetes HA集群。我遇到了F5配置问题。

我正在使用自签名证书，并将apiserver.crt和apiserver.key传递给负载均衡器。

由于某些原因，kubeadm初始化脚本失败并显示以下错误：

[apiclient] All control plane components are healthy after 33.083159 seconds
I0805 10:09:11.335063    1875 uploadconfig.go:109] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0805 10:09:11.340266    1875 request.go:947] Request Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","creationTimestamp":null},"data":{"ClusterConfiguration":"apiServer:\n  certSANs:\n  - $F5_LOAD_BALANCER_VIP\n  extraArgs:\n    authorization-mode: Node,RBAC\n  timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta2\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrolPlaneEndpoint: $F5_LOAD_BALANCER_VIP:6443\ncontrollerManager: {}\ndns:\n  type: CoreDNS\netcd:\n  local:\n    dataDir: /var/lib/etcd\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVersion: v1.15.1\nnetworking:\n  dnsDomain: cluster.local\n  podSubnet: 192.168.0.0/16\n  serviceSubnet: 10.96.0.0/12\nscheduler: {}\n","ClusterStatus":"apiEndpoints:\n  lnxkbmaster02:\n    advertiseAddress: $MASTER01_IP\n    bindPort: 6443\napiVersion: kubeadm.k8s.io/v1beta2\nkind: ClusterStatus\n"}}
I0805 10:09:11.340459    1875 round_trippers.go:419] curl -k -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubeadm/v1.15.1 (linux/amd64) kubernetes/4485c6f" 'https://$F5_LOAD_BALANCER_VIP:6443/api/v1/namespaces/kube-system/configmaps'
I0805 10:09:11.342399    1875 round_trippers.go:438] POST https://$F5_LOAD_BALANCER_VIP:6443/api/v1/namespaces/kube-system/configmaps 403 Forbidden in 1 milliseconds
I0805 10:09:11.342449    1875 round_trippers.go:444] Response Headers:
I0805 10:09:11.342479    1875 round_trippers.go:447]     Content-Type: application/json
I0805 10:09:11.342507    1875 round_trippers.go:447]     X-Content-Type-Options: nosniff
I0805 10:09:11.342535    1875 round_trippers.go:447]     Date: Mon, 05 Aug 2019 08:09:11 GMT
I0805 10:09:11.342562    1875 round_trippers.go:447]     Content-Length: 285
I0805 10:09:11.342672    1875 request.go:947] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps is forbidden: User \"system:anonymous\" cannot create resource \"configmaps\" in API group \"\" in the namespace \"kube-system\"","reason":"Forbidden","details":{"kind":"configmaps"},"code":403}
error execution phase upload-config/kubeadm: error uploading the kubeadm ClusterConfiguration: unable to create ConfigMap: configmaps is forbidden: User "system:anonymous" cannot create resource "configmaps" in API group "" in the namespace "kube-system"

初始化真的很基本：

kubeadm init --config=kubeadm-config.yaml --upload-certs

这是kubeadm-config.yaml：

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "$F5_LOAD_BALANCER_VIP:6443"
networking:
  podSubnet: "192.168.0.0/16"

如果我使用HAproxy设置集群，则初始化运行平稳：

#---------------------------------------------------------------------
# kubernetes
#---------------------------------------------------------------------
frontend kubernetes
    bind        $HAPROXY_LOAD_BALANCER_IP:6443
    option      tcplog
    mode        tcp
    default_backend kubernetes-master-nodes

backend kubernetes-master-nodes
    mode    tcp
    balance roundrobin
    option  tcp-check
    server  master01.my-domain  $MASTER_01_IP:6443 check fall 3 rise 2
    server  master02.my-domain  $MASTER_02_IP:6443 check fall 3 rise 2
    server  master03.my-domain  $MASTER_03_IP:6443 check fall 3 rise 2
END

Answer 1

我的解决方案是在不使用以下配置的代理（F5）的情况下部署群集：

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "$MASTER_1_IP:6443"
networking:
  podSubnet: "192.168.0.0/16"

然后，有必要在集群上部署用于Kubernetes的F5 BIG-IP控制器，以从Kubernetes管理F5设备。详细的指南可以在这里找到：

https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/v1.10/

请注意，它还需要其他F5许可证和管理员权限。

使用kubeadm和F5作为负载均衡器设置Kubernetes HA群集

1 个答案: