在Google Kubernetes Engine上创建多集群入口时，是什么阻止设置“ instance-groups”注释？

Question

我正在尝试使用kubemci在Google Kubernetes Engine上创建多集群入口，但是当运行以下命令时，程序将无限期等待入口服务获取ingress.gcp.kubernetes.io/instance-groups注释（如在下面的输出中进行说明）。

是什么导致无法设置此注释？

输入

./kubemci create app-mci \
    --ingress=ingress.yaml \
    --gcp-project=app-prod \
    --kubeconfig=mcikubeconfig

输出

% ./kubemci create app-mci --ingress=ingress.yaml --gcp-project=app-prod --kubeconfig=clusters.yaml        
Created Ingress in cluster: gke_app-prod_europe-west4-a_app-europe-west4
Created Ingress in cluster: gke_app-prod_us-east4-a_app-us-east4
Ensuring health checks
Pod app-deployment-c99578769-xdmql matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-xgq2m matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-qms7r matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Pod app-deployment-c99578769-tsrsw matching service selectors app=app (targetport ): lacks a matching HTTP probe for use in health checks.
Path for healthcheck is /
Ensuring health check for port: {SvcName:default/app-service SvcPort:{Type:0 IntVal:80 StrVal:} NodePort:30061 Protocol:HTTP SvcTargetPort: NEGEnabled:false}
Health check mci1-hc-30061--app-mci exists already. Checking if it matches our desired health check
Desired health check exists already
Determining instance groups for cluster gke_app-prod_europe-west4-a_app-europe-west4
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
Waiting for ingress ( default : app-ingress ) to get ingress.gcp.kubernetes.io/instance-groups annotation.....
⋮

您可以看到我的配置（除了资源名称）与multi-cluster ingress guide中的配置相同：

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
spec:
  selector:
    matchLabels:
      app: app
  replicas: 2
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
        - name: app
          image: gcr.io/app-prod/app:tag
          ports:
            - containerPort: 8080

service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: app
  name: app-service
spec:
  ports:
    - port: 80
      protocol: TCP
      targetPort: 8080
      name: http
      nodePort: 30061
  selector:
    app: app
  type: NodePort

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: app-ip
    kubernetes.io/ingress.class: gce-multi-cluster
spec:
  backend:
    serviceName: app-service
    servicePort: 80

Answer 1

启用HTTP负载平衡

启用HTTP负载平衡插件，以允许load balancer controller设置ingress.gcp.kubernetes.io/instance-groups注释。

控制台

1. 编辑集群。
2. 扩展插件。
3. 启用HTTP负载平衡：

命令行

% gcloud container clusters update [CLUSTER_NAME] --update-addons HttpLoadBalancing=ENABLED

Updating ***...done.                                                                                                                                                              
Updated [https://container.googleapis.com/v1/projects/***/zones/us-east4-a/clusters/***].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-east4-a/***?project=***

查看集群配置：

% gcloud container clusters describe [CLUSTER_NAME]

# ENABLED
addonsConfig:
  httpLoadBalancing: {}

# DISABLED
addonsConfig:
  httpLoadBalancing:
    disabled: true

配置服务

确保正确配置了多集群入口中使用的后端服务。

服务必须：

• 在所有群集中都具有相同的名称。
• 在所有集群中都位于相同的命名空间中。
• 属于NodePort类型。

• 对所有群集使用相同的端口号。

  — Setting up a multi-cluster Ingress，Google

信用

• Nikhil Jindal的见识。
• 伊万（Ivan）筹集了this issue。