Kubernetes运行状况检查因自定义Nginx Web服务器配置而失败

时间:2020-05-01 12:47:16

标签: docker nginx kubernetes google-kubernetes-engine basic-authentication

我的健康检查因以下设置而失败。

nginx.conf

user                            root;
worker_processes                auto;

error_log                       /var/log/nginx/error.log warn;

events {
    worker_connections          1024;
}

http {
    server {
        listen                  80;
        server_name             subdomain.domain.com
        auth_basic              "Restricted";
        auth_basic_user_file    /etc/nginx/.htpasswd;
    }
    server {
        listen                  80;
        auth_basic              off;
    }
    server {
        listen                  2222;
        auth_basic              off;
        location /healthz {
            return 200;
        }
    }
}

DOCKERFILE

FROM nginx:alpine
COPY index.html /usr/share/nginx/html/index.html
VOLUME /usr/share/nginx/html
COPY /server/nginx.conf /etc/nginx/
COPY /server/htpasswd /etc/nginx/.htpasswd
CMD ["nginx", "-g", "daemon off;"]
EXPOSE 80
EXPOSE 2222

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: my-namespace
  labels:
    app: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-app
          image: gcr.io/GOOGLE_CLOUD_PROJECT/my-app
          ports:
            - containerPort: 80
            - containerPort: 2222
          livenessProbe:
            httpGet:
              path: /healthz
              port: 2222
          readinessProbe:
            httpGet:
              path: /healthz
              port: 2222

当我删除nginx.conf中的“ server_name”行并删除第二个服务器块时,它绝对有效。 这可能是入口/负载平衡器的问题,因为我不知道更新需要多长时间(昨天,我经历了一个健康的吊舱,几分钟后变得不健康)。使用Google自己的入口控制器(不是NGINX入口!)在Google Kubernetes Engine(GKE)上运行它

我在做什么错了?

1 个答案:

答案 0 :(得分:1)

问题在于GKE的负载均衡器会执行自己的运行状况检查。这些默认情况下查看/,并期望返回200。只有在部署/吊舱中的运行状况检查声明了其他路径时,负载均衡器的运行状况检查才会选择这些路径。

在应用入口YAML之后,将配置负载均衡器。只要负载均衡器运行,部署或入口中影响负载均衡器的任何更改都不会被接受。这意味着我必须先删除负载平衡器,然后再应用部署,服务和入口YAML(然后inress将自动设置负载平衡器)。除了删除负载均衡器之外,您还可以手动输入正确的路径(并等待几分钟)。

由于负载均衡器似乎在每个打开的端口上进行了运行状况检查,因此我删除了2222端口,并将位置/ healthz添加到nginx中每个端口为80的服务器块中,并将auth_basic关闭。

请参阅:https://cloud.google.com/load-balancing/docs/health-check-conceptshttps://stackoverflow.com/a/61222826/2534357https://stackoverflow.com/a/38511357/2534357

新的nginx.conf

user                            root;
worker_processes                auto;

error_log                       /var/log/nginx/error.log warn;

events {
    worker_connections          1024;
}

http {
    server {
        listen                  80;
        server_name             subdomain1.domain.com;
        root                    /usr/share/nginx/html;
        index                   index.html;
        auth_basic              "Restricted";
        auth_basic_user_file    /etc/nginx/.htpasswd_subdomain1;
        location /healthz {
            auth_basic          off;
            allow               all;
            return              200;
        }
    }
    server {
        listen                  80;
        server_name             subdomain2.domain.com;
        root                    /usr/share/nginx/html;
        index                   index.html;
        auth_basic              "Restricted";
        auth_basic_user_file    /etc/nginx/.htpasswd_subdomain2;
        location /healthz {
            auth_basic          off;
            allow               all;
            return              200;
        }
    }
    server {
        listen                  80;
        server_name             domain.com www.domain.com;
        root                    /usr/share/nginx/html;
        index                   index.html;
        auth_basic              "Restricted";
        auth_basic_user_file    /etc/nginx/.htpasswd_domain;
        location /healthz {
            auth_basic          off;
            allow               all;
            return              200;
        }
    }
    ## next block probably not necessary
    server {
        listen                  80;
        auth_basic              off;
        location /healthz {
            return              200;
        }
    }
}

我的新Deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: my-namespace
  labels:
    app: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-app
          image: gcr.io/GOOGLE_CLOUD_PROJECT/my-app
          ports:
            - containerPort: 80
          livenessProbe:
            httpGet:
              path: /healthz
              port: 80
          readinessProbe:
            httpGet:
              path: /healthz
              port: 80