Kubernetes滚动更新无法正常工作

时间:2018-06-13 07:38:13

标签: kubernetes kops

我有2个kubernetes安装用于不同的项目,尽管我可以看到在相关的区域中具有相同的配置但是2执行不同的滚动更新。

两者都使用kops安装在AWS上。

系统1(k8s v1.7.0) - 使用k8s web gui杀死部署中的pod,首先创建新pod,然后一旦运行将终止旧pod。没有停机时间。

系统2(k8s v1.8.4) - 使用k8s web gui杀死部署中的pod,立即终止旧pod,然后创建新pod。导致短暂的停机时间。

关于他们为什么表现不同的任何建议或想法,以及如何在终止旧广告之前让系统2创建新广告?

系统1部署

{
  "kind": "Deployment",
  "apiVersion": "extensions/v1beta1",
  "metadata": {
    "name": "proxy-deployment",
    "namespace": "namespace",
    "selfLink": "/apis/extensions/v1beta1/namespaces/namespace/deployments/proxy-deployment",
    "uid": "d12778ba-8950-11e7-9e69-12f38e55b21a",
    "resourceVersion": "31538492",
    "generation": 7,
    "creationTimestamp": "2017-08-25T04:49:45Z",
    "labels": {
      "app": "proxy"
    },
    "annotations": {
      "deployment.kubernetes.io/revision": "6",
      "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"apps/v1beta1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"name\":\"proxy-deployment\",\"namespace\":\"namespace\"},\"spec\":{\"replicas\":2,\"template\":{\"metadata\":{\"labels\":{\"app\":\"proxy\"}},\"spec\":{\"containers\":[{\"image\":\"xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/nginx-proxy-xxxxxx:latest\",\"name\":\"proxy-ctr\",\"ports\":[{\"containerPort\":80},{\"containerPort\":8080}]}]}}}}\n"
    }
  },
  "spec": {
    "replicas": 1,
    "selector": {
      "matchLabels": {
        "app": "proxy"
      }
    },
    "template": {
      "metadata": {
        "creationTimestamp": null,
        "labels": {
          "app": "proxy",
          "date": "1522386390"
        }
      },
      "spec": {
        "containers": [
          {
            "name": "proxy-ctr",
            "image": "xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/nginx-proxy-xxxxxx:latest",
            "ports": [
              {
                "containerPort": 80,
                "protocol": "TCP"
              },
              {
                "containerPort": 8080,
                "protocol": "TCP"
              }
            ],
            "resources": {},
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "Always"
          }
        ],
        "restartPolicy": "Always",
        "terminationGracePeriodSeconds": 30,
        "dnsPolicy": "ClusterFirst",
        "securityContext": {},
        "schedulerName": "default-scheduler"
      }
    },
    "strategy": {
      "type": "RollingUpdate",
      "rollingUpdate": {
        "maxUnavailable": "25%",
        "maxSurge": "25%"
      }
    },
    "revisionHistoryLimit": 2,
    "progressDeadlineSeconds": 600
  },
  "status": {
    "observedGeneration": 7,
    "replicas": 1,
    "updatedReplicas": 1,
    "readyReplicas": 1,
    "availableReplicas": 1,
    "conditions": [
      {
        "type": "Progressing",
        "status": "True",
        "lastUpdateTime": "2018-03-30T05:03:01Z",
        "lastTransitionTime": "2017-08-25T04:49:45Z",
        "reason": "NewReplicaSetAvailable",
        "message": "ReplicaSet \"proxy-deployment-1457650622\" has successfully progressed."
      },
      {
        "type": "Available",
        "status": "True",
        "lastUpdateTime": "2018-06-01T06:55:12Z",
        "lastTransitionTime": "2018-06-01T06:55:12Z",
        "reason": "MinimumReplicasAvailable",
        "message": "Deployment has minimum availability."
      }
    ]
  }
}

系统2部署

{
  "kind": "Deployment",
  "apiVersion": "extensions/v1beta1",
  "metadata": {
    "name": "prodefault-deployment",
    "namespace": "namespace",
    "selfLink": "/apis/extensions/v1beta1/namespaces/namespace/deployments/prodefault-deployment",
    "uid": "a80528c8-eb79-11e7-9364-068125440f70",
    "resourceVersion": "25203392",
    "generation": 10,
    "creationTimestamp": "2017-12-28T02:49:00Z",
    "labels": {
      "app": "prodefault"
    },
    "annotations": {
      "deployment.kubernetes.io/revision": "7",
      "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"apps/v1beta1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"name\":\"prodefault-deployment\",\"namespace\":\"namespace\"},\"spec\":{\"replicas\":1,\"strategy\":{\"rollingUpdate\":{\"maxSurge\":\"25%\",\"maxUnavailable\":\"25%\"},\"type\":\"RollingUpdate\"},\"template\":{\"metadata\":{\"labels\":{\"app\":\"prodefault\"}},\"spec\":{\"containers\":[{\"image\":\"xxxxxxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/xxxxxxxxxxx-pro-default:latest\",\"livenessProbe\":{\"httpGet\":{\"path\":\"/healthchk\",\"port\":80},\"initialDelaySeconds\":120,\"periodSeconds\":15,\"timeoutSeconds\":1},\"name\":\"prodefault-ctr\",\"ports\":[{\"containerPort\":80}],\"readinessProbe\":{\"httpGet\":{\"path\":\"/healthchk\",\"port\":80},\"initialDelaySeconds\":5,\"periodSeconds\":2,\"timeoutSeconds\":3},\"resources\":{\"limits\":{\"cpu\":\"1\",\"memory\":\"1024Mi\"},\"requests\":{\"cpu\":\"150m\",\"memory\":\"256Mi\"}},\"volumeMounts\":[{\"mountPath\":\"/var/www/html/homes\",\"name\":\"efs-pvc\"},{\"mountPath\":\"/var/xero\",\"name\":\"xero-key\",\"readOnly\":true},{\"mountPath\":\"/var/gcal\",\"name\":\"gcal-json\",\"readOnly\":true}]}],\"volumes\":[{\"name\":\"efs-pvc\",\"persistentVolumeClaim\":{\"claimName\":\"tio-pv-claim-homes\"}},{\"name\":\"xero-key\",\"secret\":{\"secretName\":\"xero-key\"}},{\"name\":\"gcal-json\",\"secret\":{\"secretName\":\"gcaljson\"}}]}}}}\n"
    }
  },
  "spec": {
    "replicas": 1,
    "selector": {
      "matchLabels": {
        "app": "prodefault"
      }
    },
    "template": {
      "metadata": {
        "creationTimestamp": null,
        "labels": {
          "app": "prodefault"
        }
      },
      "spec": {
        "volumes": [
          {
            "name": "efs-pvc",
            "persistentVolumeClaim": {
              "claimName": "tio-pv-claim-homes"
            }
          },
          {
            "name": "xero-key",
            "secret": {
              "secretName": "xero-key",
              "defaultMode": 420
            }
          },
          {
            "name": "gcal-json",
            "secret": {
              "secretName": "gcaljson",
              "defaultMode": 420
            }
          }
        ],
        "containers": [
          {
            "name": "prodefault-ctr",
            "image": "xxxxxxxxxxxx.dkr.ecr.us-west-2.amazonaws.com/xxxxxxxxxxx-pro-default:latest",
            "ports": [
              {
                "containerPort": 80,
                "protocol": "TCP"
              }
            ],
            "resources": {
              "limits": {
                "cpu": "1",
                "memory": "1Gi"
              },
              "requests": {
                "cpu": "150m",
                "memory": "256Mi"
              }
            },
            "volumeMounts": [
              {
                "name": "efs-pvc",
                "mountPath": "/var/www/html/homes"
              },
              {
                "name": "xero-key",
                "readOnly": true,
                "mountPath": "/var/xero"
              },
              {
                "name": "gcal-json",
                "readOnly": true,
                "mountPath": "/var/gcal"
              }
            ],
            "livenessProbe": {
              "httpGet": {
                "path": "/healthchk",
                "port": 80,
                "scheme": "HTTP"
              },
              "initialDelaySeconds": 120,
              "timeoutSeconds": 1,
              "periodSeconds": 15,
              "successThreshold": 1,
              "failureThreshold": 3
            },
            "readinessProbe": {
              "httpGet": {
                "path": "/healthchk",
                "port": 80,
                "scheme": "HTTP"
              },
              "initialDelaySeconds": 5,
              "timeoutSeconds": 3,
              "periodSeconds": 2,
              "successThreshold": 1,
              "failureThreshold": 3
            },
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "Always"
          }
        ],
        "restartPolicy": "Always",
        "terminationGracePeriodSeconds": 30,
        "dnsPolicy": "ClusterFirst",
        "securityContext": {},
        "schedulerName": "default-scheduler"
      }
    },
    "strategy": {
      "type": "RollingUpdate",
      "rollingUpdate": {
        "maxUnavailable": "25%",
        "maxSurge": "25%"
      }
    },
    "revisionHistoryLimit": 2,
    "progressDeadlineSeconds": 600
  },
  "status": {
    "observedGeneration": 10,
    "replicas": 1,
    "updatedReplicas": 1,
    "readyReplicas": 1,
    "availableReplicas": 1,
    "conditions": [
      {
        "type": "Progressing",
        "status": "True",
        "lastUpdateTime": "2018-01-15T06:07:52Z",
        "lastTransitionTime": "2017-12-28T03:00:16Z",
        "reason": "NewReplicaSetAvailable",
        "message": "ReplicaSet \"prodefault-deployment-9685f46d4\" has successfully progressed."
      },
      {
        "type": "Available",
        "status": "True",
        "lastUpdateTime": "2018-06-13T07:12:41Z",
        "lastTransitionTime": "2018-06-13T07:12:41Z",
        "reason": "MinimumReplicasAvailable",
        "message": "Deployment has minimum availability."
      }
    ]
  }
}

1 个答案:

答案 0 :(得分:1)

我注意到两个pod都定义了以下滚动更新策略:

"strategy": {
  "type": "RollingUpdate",
  "rollingUpdate": {
    "maxUnavailable": "25%",
    "maxSurge": "25%"
  }
},

通过这种方式,它应该是在正常滚动更新中通过“设置图像”创建新窗格后终止旧窗格。或者' kubectl适用'。

因此,两个系统之间的不同行为可能来自仪表板。我猜你在两个系统中运行不同版本的仪表板,因为根据仪表板的兼容性指标,kubernetes v1.7需要支持仪表板1.7,而kubernetes v1.8需要仪表板1.8才能支持。也许版本不同的仪表板对待'杀死pod'作为不同的行动,我不知道。

或者,如果您在v1.8系统中运行dashboard 1.7,请先尝试升级您的dashbord。

最后,不要使用' kill pod'做滚动更新。