头盔和GKE版本:
掌舵:2.9.1
Kubernetes:1.10.6-gke.2
TLS
让我们加密安装。
发生了什么事
Gitlab Runner和Gitlab网站:突然收到502错误的网关。
https://minio.mydomain.com有效。
> kubectl get pods mydomain-gitaly-0 1/1 Running 0 13h mydomain-gitlab-runner-55fd547dd4-db5hz 0/1 CrashLoopBackOff 69 13h mydomain-gitlab-shell-84f55fbf74-nwsch 1/1 Running 0 13h mydomain-gitlab-shell-84f55fbf74-rct44 1/1 Running 0 13h mydomain-minio-578c894db7-65rx8 1/1 Running 0 13h mydomain-nginx-ingress-controller-59448dc446-grlzs 1/1 Running 0 13h mydomain-nginx-ingress-controller-59448dc446-vvv4m 1/1 Running 0 13h mydomain-nginx-ingress-controller-59448dc446-wtjmq 1/1 Running 0 13h mydomain-nginx-ingress-default-backend-84c7d74465-9x4ll 1/1 Running 0 13h mydomain-nginx-ingress-default-backend-84c7d74465-b48lk 1/1 Running 0 13h mydomain-postgresql-6d8bf8d756-kc2sg 2/2 Running 0 13h mydomain-prometheus-server-d789bcc55-kjvks 2/2 Running 0 13h mydomain-redis-66b758d87f-fpfr6 2/2 Running 0 13h mydomain-registry-68c446cc59-p474n 1/1 Running 0 13h mydomain-registry-68c446cc59-x44cx 1/1 Running 0 13h mydomain-sidekiq-all-in-1-559b68d649-q65jq 1/1 Running 0 13h mydomain-task-runner-6674f8fc98-dk4gw 1/1 Running 0 13h mydomain-unicorn-5d944d8968-fxwjn 1/1 Running 0 13h mydomain-unicorn-5d944d8968-mtwmz 1/1 Running 0 13h certmanager-mydomain-56fb8bfb87-jbz97 2/2 Running 0 13h
> kubectl logs mydomain-gitlab-runner-55fd547dd4-db5hz WARNING: Running in user-mode. WARNING: The user-mode requires you to manually start builds processing: WARNING: $ gitlab-runner run WARNING: Use sudo for system-mode: WARNING: $ sudo gitlab-runner... ERROR: Registering runner... failed runner=D9b2BixA status=502 Bad Gateway PANIC: Failed to register this runner. Perhaps you are having network problems
> kubectl logs mydomain-nginx-ingress-controller-59448dc446-grlzs .. 2018/09/17 02:59:23 [error] 775#775: *83 connect() failed (111: Connection refused) while connecting to upstream, client: 10.44.1.14, server: gitlab.mydomain.com, request: "POST /api/v4/runners HTTP/1.1", upstream: "http://10.44.2.7:8181/api/v4/runners", host: "gitlab.mydomain.com" 2018/09/17 02:59:23 [error] 775#775: *83 connect() failed (111: Connection refused) while connecting to upstream, client: 10.44.1.14, server: gitlab.mydomain.com, request: "POST /api/v4/runners HTTP/1.1", upstream: "http://10.44.1.21:8181/api/v4/runners", host: "gitlab.mydomain.com" 10.44.1.14 - [10.44.1.14] - - [17/Sep/2018:02:59:23 +0000] "POST /api/v4/runners HTTP/1.1" 502 166 "-" "gitlab-runner 10.3.0 (10-3-stable; go1.8.5; linux/amd64)" 619 0.002 [default-mydomain-unicorn-8181] 10.44.2.7:8181, 10.44.1.21:8181 0, 0 0.003, 0.000 502, 502 W0917 03:02:15.709349 6 reflector.go:341] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:140: watch of *v1.Endpoints ended with: too old resource version: 5778376 (5779396) W0917 03:17:52.790383 6 reflector.go:341] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:140: watch of *v1.Endpoints ended with: too old resource version: 5780911 (5782212) ..