TLS无法在Kubernetes中与LoadBalancer支持的服务一起使用

时间：2019-08-06 18:47:28

标签： amazon-web-services kubernetes kubectl kubernetes-ingress kops

我试图通过将服务类型创建为负载平衡器来公开群集中的应用程序。这样做的原因是我希望该应用具有单独的通信渠道。我有一个KOPS集群。我想使用AWS的网络负载平衡器，以便获得静态IP。当我使用端口80映射到该应用程序正在运行的端口来创建服务时，一切正常，但是当我尝试添加端口443时，它就超时了。

这是有效的配置-

apiVersion: v1
metadata:
  name: abc
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
  labels:
    app: abc
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 9050
  selector:
    app: abc
  type: LoadBalancer

一旦我在配置文件中添加TLS支持并进行部署。与负载均衡器的连接超时。如何为负载均衡器添加TLS支持？ 我想通过服务而不是通过入口做到这一点。 此配置不适用于我，当我将链接粘贴到浏览器中时，它会超时。

kind: Service
apiVersion: v1
metadata:
  name: abc
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxxx
  labels:
    app: abc
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 443
    protocol: TCP
    targetPort: 9050
  selector:
    app: abc
  type: LoadBalancer

2 个答案:

答案 0 :(得分：1)

您可以使用tls和ssl终止

   apiVersion: v1
    kind: Service
    metadata:
      name: test-service
      annotations:
        # Note that the backend talks over HTTP.
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
        # TODO: Fill in with the ARN of your certificate.
        service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:{region}:{user id}:certificate/{id}
        # Only run SSL on the port named "https" below.
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    spec:
      selector:
        app: test-pod
      ports:
      - name: http
        port: 80
        targetPort: 8080
      - name: https
        port: 443
        targetPort: 8080
      type: LoadBalancer

您可以在aws证书管理器中添加tls证书，并将证书的arn地址用于kubernetes服务。

就像您可以终止https连接并仅使用HTTP一样。

您还可以查看以下内容：https://aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/ 

https://github.com/kubernetes/kubernetes/issues/73297

编辑：1 

service.beta.kubernetes.io/aws-load-balancer-type: nlb

如果不起作用，请尝试根据您的负载均衡器类型添加此注释。

答案 1 :(得分：0)

您现在可以使用NLB和SSL终止（NLB中的https>服务中的http）部署入口。 终于找到了一个对我有用的解决方案，您可以尝试部署以下ingress.yaml（确保在“部署”部分下更新您的cert ARN）：

    ---
# Source: nginx-ingress/templates/controller-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
---
# Source: nginx-ingress/templates/default-backend-serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-backend
---
# Source: nginx-ingress/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    component: "controller"
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-controller
data:
  server-snippet: |
    listen 8000;
    if ( $server_port = 80 ) {
       return 308 https://$host$request_uri;
    }
  ssl-redirect: "false"
---
# Source: nginx-ingress/templates/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - update
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
---
# Source: nginx-ingress/templates/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress
subjects:
  - kind: ServiceAccount
    name: nginx-ingress
    namespace: ingress-nginx
---
# Source: nginx-ingress/templates/controller-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
rules:
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - update
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - "networking.k8s.io" # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - ingress-controller-leader-nginx
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - create
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
---
# Source: nginx-ingress/templates/controller-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress
subjects:
  - kind: ServiceAccount
    name: nginx-ingress
    namespace: ingress-nginx
---
# Source: nginx-ingress/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
  namespace: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:##REPLACE WITH YOUR CERT ARN"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    component: "controller"
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-controller
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: special
  selector:
    app: nginx-ingress
    release: nginx-ingress
    app.kubernetes.io/component: controller
  type: "LoadBalancer"
---
# Source: nginx-ingress/templates/default-backend-service.yaml
apiVersion: v1
kind: Service
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    component: "default-backend"
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-default-backend
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
  selector:
    app: nginx-ingress
    release: nginx-ingress
    app.kubernetes.io/component: default-backend
  type: "ClusterIP"
---
# Source: nginx-ingress/templates/controller-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
    app.kubernetes.io/component: controller
  name: nginx-ingress-controller
  annotations:
    {}
spec:
  selector:
    matchLabels:
      app: nginx-ingress
      release: nginx-ingress
  replicas: 1
  revisionHistoryLimit: 10
  strategy:
    {}
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app: nginx-ingress
        release: nginx-ingress
        component: "controller"
        app.kubernetes.io/component: controller
    spec:
      dnsPolicy: ClusterFirst
      containers:
        - name: nginx-ingress-controller
          image: "quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0"
          imagePullPolicy: "IfNotPresent"
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/nginx-ingress-default-backend
            - --publish-service=$(POD_NAMESPACE)/nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/nginx-ingress-controller
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            capabilities:
                drop:
                - ALL
                add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          livenessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: special
              containerPort: 8000
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          resources:
            {}
      hostNetwork: false
      serviceAccountName: nginx-ingress
      terminationGracePeriodSeconds: 60
---
# Source: nginx-ingress/templates/default-backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: ingress-nginx
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.38.0
    heritage: Helm
    release: nginx-ingress
    app.kubernetes.io/component: default-backend
  name: nginx-ingress-default-backend
spec:
  selector:
    matchLabels:
      app: nginx-ingress
      release: nginx-ingress
  replicas: 1
  revisionHistoryLimit: 10
  template:
    metadata:
      labels:
        app: nginx-ingress
        release: nginx-ingress
        app.kubernetes.io/component: default-backend
    spec:
      containers:
        - name: nginx-ingress-default-backend
          image: "k8s.gcr.io/defaultbackend-amd64:1.5"
          imagePullPolicy: "IfNotPresent"
          args:
          securityContext:
            runAsUser: 65534
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 0
            periodSeconds: 5
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 6
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          resources:
            {}
      serviceAccountName: nginx-ingress-backend
      terminationGracePeriodSeconds: 60
相关问题
最新问题