在群集上启用了mTLS的AWS EKS上,无法通过Istio Gateway(以ELB运行)通过HTTPS访问pod

时间:2019-02-22 14:32:44

标签: amazon-web-services kubernetes amazon-elb kubernetes-ingress istio

我们已通过Helm在新启动的EKS集群上安装了Istio 1.0.5。

helm install \
    --wait \
    --name istio \
    --namespace istio-system \
    --set tracing.enabled=true \
    --set kiali.enabled=true \
    --set grafana.enabled=true \
    --set global.mtls.enabled=true \
    --set servicegraph.enabled=true \
    --set global.proxy.includeIPRanges="10.10.0.0/16\,172.20.0.0/16" \
    install/kubernetes/helm/istio 

(the IP ranges are pod and service CIDR ranges of the cluster)

这是我们要实现的目标: istio ingress gateway

为了使ACM证书能够附加到由istio-ingressgateway启动的ELB,向istio-ingressgateway服务添加了以下注释:

service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-1:ACCOUNT:certificate/MY_CERT
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https

正在按预期创建Ingress Gateway负载平衡器服务:

enter image description here enter image description here

我们正在以下链接上实现上述情况:

https://doc.istio.cn/en/docs/examples/advanced-gateways/ingress-sni-passthrough/

证书和机密信息是根据上述Istio文档创建的。服务和部署资源的创建如下:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  labels:
    run: my-nginx
spec:
  ports:
  - port: 443
    protocol: TCP
  selector:
    run: my-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 443
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx
          readOnly: true
        - name: nginx-server-certs
          mountPath: /etc/nginx-server-certs
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-configmap
      - name: nginx-server-certs
        secret:
          secretName: nginx-server-certs

网关和VirtualService资源的创建如下:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: PASSTHROUGH
    hosts:
    - nginx.example.com

---

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - nginx.example.com
  gateways:
  - mygateway
  tls:
  - match:
    - port: 443
      sni_hosts:
      - nginx.example.com
    route:
    - destination:
        host: my-nginx
        port:
          number: 443

正在按预期创建资源:

enter image description here

我们可以从Pod内的Nginx容器获得响应:

curl --insecure --verbose https://localhost

enter image description here

我们还可以通过Nginx容器的服务从另一个容器获得响应:

curl --insecure --verbose https://my-nginx

enter image description here

但是,当我们尝试通过Istio入口网关访问nginx pod时,该服务将无法访问:

curl --insecure --verbose https://a1a99cb2d34f711e9865b0295f80a9c0-303710594.us-east-1.elb.amazonaws.com

enter image description here

为ELB启用访问日志后,这是相关的日志行:

2019-02-22T12:36:08.960931Z a1a99cb2d34f711e9865b0295f80a9c0 3.84.67.219:56786 - -1 -1 -1 503 0 0 0 "GET https://a1a99cb2d34f711e9865b0295f80a9c0-303710594.us-east-1.elb.amazonaws.com:443/ HTTP/1.1" "curl/7.55.1" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2

我们进入istio-ingressgateway窗格中,并尝试通过其服务名称(my-nginx),服务ip(172.20.198.12),服务dns(my-nginx.default.svc.cluster.local)访问nginx窗格。以及pod ip(10.10.157.58)。 但是它们都无法访问:

enter image description here

我们需要创建任何额外的VirtualService或ServiceEntry吗? 当我们尝试通过Istio入口网关访问端口80上公开的部署时,我们能够成功命中Pod。 仅当我们尝试在Istio ingressgateway服务上启用HTTPS时,才会出现此问题。 是否因为启用了mTLS而发生这种情况? 请帮助我们找到有关如何进行操作的指示。

0 个答案:

没有答案