我的部署不知何故消失了,是否有任何地方可以追踪“已删除”操作?

时间:2019-06-12 11:44:22

标签: kubernetes

我在Kubernetes集群中部署了部署nginx,如下所示。 部署如下所示:

[root@iZwz97twzbvh62zh76pk8zZ zzx]# k get deployment nginx
NAME    DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx   2         2         2            1           7s

部署模板如下:

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.17.0
        ports:
        - containerPort: 80

以某种方式部署nginx消失了,我想知道是否有任何地方可以跟踪“已删除”操作?

2 个答案:

答案 0 :(得分:1)

必须有另一个与用户操作相关的审核日志,这些日志最初会触发您的Kubernetes集群上的“删除部署”操作。

您粘贴的审核日志只是此删除操作链的一部分,来自系统组件:kubelet,负责删除Pod-> ReplicaSet,它属于您的Deployment资源。

因此,查找包含以下信息的审核日志:

  authorizationInfo: [
   0: {
    granted:  true     
    permission:  "io.k8s.extensions.v1beta1.deployments.delete"
    resource:  "extensions/v1beta1/namespaces/default/deployments/nginx"     
   }
 ]

答案 1 :(得分:0)

我的一位同事告诉我,所有的资源删除操作都将在Kube-Apiserver日志中标记出来,因此在/var/log/kubernetes-2019-06-12T09-34-21.649.audit中,我找到了一些线索。

{
  "kind": "Event",
  "apiVersion": "audit.k8s.io/v1beta1",
  "metadata": {
    "creationTimestamp": "2019-06-12T08:48:36Z"
  },
  "level": "RequestResponse",
  "timestamp": "2019-06-12T08:48:36Z",
  "auditID": "b67824ac-dbb4-4e95-a0c6-8cdc31c8914a",
  "stage": "ResponseComplete",
  "requestURI": "/api/v1/namespaces/default/pods/nginx-deployment-5d599789c6-kvcdg",
  "verb": "delete",
  "user": {
    "username": "system:node:izwz9gvzqgre2jnudydwigz",
    "groups": [
      "system:nodes",
      "system:authenticated"
    ]
  },
  "sourceIPs": [
    "172.18.252.140"
  ],
  "userAgent": "kubelet/v1.12.6 (linux/amd64) kubernetes/01a904e",
  "objectRef": {
    "resource": "pods",
    "namespace": "default",
    "name": "nginx-deployment-5d599789c6-kvcdg",
    "apiVersion": "v1"
  },
  "responseStatus": {
    "metadata": {

    },
    "code": 200
  },
  "requestObject": {
    "kind": "DeleteOptions",
    "apiVersion": "v1",
    "gracePeriodSeconds": 0,
    "preconditions": {
      "uid": "c20699d5-8cec-11e9-8895-00163e049a3b"
    }
  },
  "responseObject": {
    "kind": "Pod",
    "apiVersion": "v1",
    "metadata": {
      "name": "nginx-deployment-5d599789c6-kvcdg",
      "generateName": "nginx-deployment-5d599789c6-",
      "namespace": "default",
      "selfLink": "/api/v1/namespaces/default/pods/nginx-deployment-5d599789c6-kvcdg",
      "uid": "c20699d5-8cec-11e9-8895-00163e049a3b",
      "resourceVersion": "3272546",
      "creationTimestamp": "2019-06-12T08:33:30Z",
      "deletionTimestamp": "2019-06-12T08:48:25Z",
      "deletionGracePeriodSeconds": 0,
      "labels": {
        "app": "nginx",
        "pod-template-hash": "5d599789c6"
      },
      "ownerReferences": [
        {
          "apiVersion": "apps/v1",
          "kind": "ReplicaSet",
          "name": "nginx-deployment-5d599789c6",
          "uid": "5c4d4ccb-8cec-11e9-8895-00163e049a3b",
          "controller": true,
          "blockOwnerDeletion": true
        }
      ]
    },
    "spec": {
      "volumes": [
        {
          "name": "default-token-k2qcm",
          "secret": {
            "secretName": "default-token-k2qcm",
            "defaultMode": 420
          }
        }
      ],
      "containers": [
        {
          "name": "nginx",
          "image": "nginx:latest2",
          "ports": [
            {
              "containerPort": 80,
              "protocol": "TCP"
            }
          ],
          "resources": {

          },
          "volumeMounts": [
            {
              "name": "default-token-k2qcm",
              "readOnly": true,
              "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
            }
          ],
          "terminationMessagePath": "/dev/termination-log",
          "terminationMessagePolicy": "File",
          "imagePullPolicy": "IfNotPresent"
        }
      ],
      "restartPolicy": "Always",
      "terminationGracePeriodSeconds": 30,
      "dnsPolicy": "ClusterFirst",
      "serviceAccountName": "default",
      "serviceAccount": "default",
      "nodeName": "izwz9gvzqgre2jnudydwigz",
      "securityContext": {

      },
      "schedulerName": "default-scheduler",
      "tolerations": [
        {
          "key": "node.kubernetes.io/not-ready",
          "operator": "Exists",
          "effect": "NoExecute",
          "tolerationSeconds": 300
        },
        {
          "key": "node.kubernetes.io/unreachable",
          "operator": "Exists",
          "effect": "NoExecute",
          "tolerationSeconds": 300
        }
      ],
      "priority": 0
    },
    "status": {
      "phase": "Running",
      "conditions": [
        {
          "type": "Initialized",
          "status": "True",
          "lastProbeTime": null,
          "lastTransitionTime": "2019-06-12T08:33:30Z"
        },
        {
          "type": "Ready",
          "status": "False",
          "lastProbeTime": null,
          "lastTransitionTime": "2019-06-12T08:47:25Z",
          "reason": "ContainersNotReady",
          "message": "containers with unready status: [nginx]"
        },
        {
          "type": "ContainersReady",
          "status": "False",
          "lastProbeTime": null,
          "lastTransitionTime": "2019-06-12T08:47:25Z",
          "reason": "ContainersNotReady",
          "message": "containers with unready status: [nginx]"
        },
        {
          "type": "PodScheduled",
          "status": "True",
          "lastProbeTime": null,
          "lastTransitionTime": "2019-06-12T08:33:30Z"
        }
      ],
      "hostIP": "172.18.252.140",
      "podIP": "10.254.9.162",
      "startTime": "2019-06-12T08:33:30Z",
      "containerStatuses": [
        {
          "name": "nginx",
          "state": {
            "terminated": {
              "exitCode": 0,
              "startedAt": null,
              "finishedAt": null
            }
          },
          "lastState": {
            "terminated": {
              "exitCode": 0,
              "reason": "Completed",
              "startedAt": "2019-06-12T08:45:50Z",
              "finishedAt": "2019-06-12T08:47:16Z",
              "containerID": "docker://deb7ab0dda23d005e396a3cdfc6ff62347b2807041805436cc31d0c503e54e6f"
            }
          },
          "ready": false,
          "restartCount": 1,
          "image": "nginx:1.17.0",
          "imageID": "docker-pullable://nginx@sha256:bdbf36b7f1f77ffe7bd2a32e59235dff6ecf131e3b6b5b96061c652f30685f3a",
          "containerID": "docker://deb7ab0dda23d005e396a3cdfc6ff62347b2807041805436cc31d0c503e54e6f"
        }
      ],
      "qosClass": "BestEffort"
    }
  },
  "requestReceivedTimestamp": "2019-06-12T08:48:36.444193Z",
  "stageTimestamp": "2019-06-12T08:48:36.452899Z",
  "annotations": {
    "authorization.k8s.io/decision": "allow",
    "authorization.k8s.io/reason": ""
  }
}

但是我仍然不知道为什么删除了部署。