Kubernetes Pod因CrashLoopBackOff失败

时间:2017-06-22 14:46:00

标签: docker repository kubernetes kubectl

我关注this guide以使用minikube设置pod并从托管在以下位置的私有存储库中提取图像:hub.docker.com

当尝试设置一个pod来拉动图像时,我看到" CrashLoopBackoff"

pod config:

apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
    - name: private-reg-container
      image: ha/prod:latest
  imagePullSecrets:
    - name: regsecret

输出" get pod"

kubectl get pod private-reg
NAME          READY     STATUS             RESTARTS   AGE
private-reg   0/1       CrashLoopBackOff   5          4m

据我所知,图像没有问题,如果我手动拉动它们并运行它们,它们就能正常工作。

(你可以看到"成功提取图片" ha / prod:latest")

如果我将通用图像推送到像centos这样的存储库并尝试使用pod来拉动并运行它,也会发生这个问题。

此外,秘密似乎工作正常,我可以看到"拉"计入私人存储库。

以下是命令的输出:

kubectl描述pods private-reg:

[~]$ kubectl describe pods private-reg
Name:       private-reg
Namespace:  default
Node:       minikube/192.168.99.100
Start Time: Thu, 22 Jun 2017 17:13:24 +0300
Labels:     <none>
Annotations:    <none>
Status:     Running
IP:     172.17.0.5
Controllers:    <none>
Containers:
  private-reg-container:
    Container ID:   docker://1aad64750d0ba9ba826fe4f12c8814f7db77293078f8047feec686fcd8f90132
    Image:      ha/prod:latest
    Image ID:       docker://sha256:7335859e2071af518bcd0e2f373f57c1da643bb37c7e6bbc125d171ff98f71c0
    Port:       
    State:      Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Mon, 01 Jan 0001 00:00:00 +0000
      Finished:     Thu, 22 Jun 2017 17:20:04 +0300
    Ready:      False
    Restart Count:  6
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-bhvgz (ro)
Conditions:
  Type      Status
  Initialized   True 
  Ready     False 
  PodScheduled  True 
Volumes:
  default-token-bhvgz:
    Type:   Secret (a volume populated by a Secret)
    SecretName: default-token-bhvgz
    Optional:   false
QoS Class:  BestEffort
Node-Selectors: <none>
Tolerations:    <none>
Events:
  FirstSeen LastSeen    Count   From            SubObjectPath               Type        Reason      Message
  --------- --------    -----   ----            -------------               --------    ------      -------
  9m        9m      1   default-scheduler                       Normal      Scheduled   Successfully assigned private-reg to minikube
  8m        8m      1   kubelet, minikube   spec.containers{private-reg-container}  Normal      Created     Created container with id 431fecfd1d2ca03d29fd88fd6c663e66afb59dc5e86487409002dd8e9987945c
  8m        8m      1   kubelet, minikube   spec.containers{private-reg-container}  Normal      Started     Started container with id 431fecfd1d2ca03d29fd88fd6c663e66afb59dc5e86487409002dd8e9987945c
  8m        8m      1   kubelet, minikube   spec.containers{private-reg-container}  Normal      Started     Started container with id 223e6af99bb950570a27056d7401137ff9f3dc895f4f313a36e73ef6489eb61a
  8m        8m      1   kubelet, minikube   spec.containers{private-reg-container}  Normal      Created     Created container with id 223e6af99bb950570a27056d7401137ff9f3dc895f4f313a36e73ef6489eb61a
  8m        8m      2   kubelet, minikube                       Warning     FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "private-reg-container" with CrashLoopBackOff: "Back-off 10s restarting failed container=private-reg-container pod=private-reg_default(f4340638-5754-11e7-978a-08002773375c)"

  8m    8m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Started     Started container with id a98377f9aedc5947fe1dd006caddb11fb48fa2fd0bb06c20667e0c8b83a3ab6a
  8m    8m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Created     Created container with id a98377f9aedc5947fe1dd006caddb11fb48fa2fd0bb06c20667e0c8b83a3ab6a
  8m    8m  2   kubelet, minikube                       Warning FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "private-reg-container" with CrashLoopBackOff: "Back-off 20s restarting failed container=private-reg-container pod=private-reg_default(f4340638-5754-11e7-978a-08002773375c)"

  8m    8m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Started     Started container with id 261f430a80ff5a312bdbdee78558091a9ae7bc9fc6a9e0676207922f1a576841
  8m    8m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Created     Created container with id 261f430a80ff5a312bdbdee78558091a9ae7bc9fc6a9e0676207922f1a576841
  8m    7m  3   kubelet, minikube                       Warning FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "private-reg-container" with CrashLoopBackOff: "Back-off 40s restarting failed container=private-reg-container pod=private-reg_default(f4340638-5754-11e7-978a-08002773375c)"

  7m    7m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Created     Created container with id 7251ab76853d4178eff59c10bb41e52b2b1939fbee26e546cd564e2f6b4a1478
  7m    7m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Started     Started container with id 7251ab76853d4178eff59c10bb41e52b2b1939fbee26e546cd564e2f6b4a1478
  7m    5m  7   kubelet, minikube                       Warning FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "private-reg-container" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=private-reg-container pod=private-reg_default(f4340638-5754-11e7-978a-08002773375c)"

  5m    5m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Created     Created container with id 347868d03fc9730417cf234e4c96195bb9b45a6cc9d9d97973855801d52e2a02
  5m    5m  1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Started     Started container with id 347868d03fc9730417cf234e4c96195bb9b45a6cc9d9d97973855801d52e2a02
  5m    3m  12  kubelet, minikube                       Warning FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "private-reg-container" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=private-reg-container pod=private-reg_default(f4340638-5754-11e7-978a-08002773375c)"

  9m    2m      7   kubelet, minikube   spec.containers{private-reg-container}  Normal  Pulling     pulling image "ha/prod:latest"
  2m    2m      1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Started     Started container with id 1aad64750d0ba9ba826fe4f12c8814f7db77293078f8047feec686fcd8f90132
  8m    2m      7   kubelet, minikube   spec.containers{private-reg-container}  Normal  Pulled      Successfully pulled image "ha/prod:latest"
  2m    2m      1   kubelet, minikube   spec.containers{private-reg-container}  Normal  Created     Created container with id 1aad64750d0ba9ba826fe4f12c8814f7db77293078f8047feec686fcd8f90132
  8m    <invalid>   40  kubelet, minikube   spec.containers{private-reg-container}  Warning BackOff     Back-off restarting failed container
  2m    <invalid>   14  kubelet, minikube                       Warning FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "private-reg-container" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=private-reg-container pod=private-reg_default(f4340638-5754-11e7-978a-08002773375c)"

以下是命令的输出:

kubectl --v = 8 logs private-reg

I0622 17:35:01.043739   15981 cached_discovery.go:71] returning cached discovery info from /home/demo/.kube/cache/discovery/192.168.99.100_8443/apps/v1beta1/serverresources.json
I0622 17:35:01.043951   15981 cached_discovery.go:71] returning cached discovery info from /home/demo/.kube/cache/discovery/192.168.99.100_8443/v1/serverresources.json
I0622 17:35:01.045061   15981 cached_discovery.go:118] returning cached discovery info from /home/demo/.kube/cache/discovery/192.168.99.100_8443/servergroups.json
I0622 17:35:01.045175   15981 round_trippers.go:395] GET https://192.168.99.100:8443/api/v1/namespaces/default/pods/private-reg
I0622 17:35:01.045182   15981 round_trippers.go:402] Request Headers:
I0622 17:35:01.045187   15981 round_trippers.go:405]     Accept: application/json, */*
I0622 17:35:01.045191   15981 round_trippers.go:405]     User-Agent: kubectl/v1.6.6 (linux/amd64) kubernetes/7fa1c17
I0622 17:35:01.072863   15981 round_trippers.go:420] Response Status: 200 OK in 27 milliseconds
I0622 17:35:01.072900   15981 round_trippers.go:423] Response Headers:
I0622 17:35:01.072921   15981 round_trippers.go:426]     Content-Type: application/json
I0622 17:35:01.072930   15981 round_trippers.go:426]     Content-Length: 2216
I0622 17:35:01.072936   15981 round_trippers.go:426]     Date: Thu, 22 Jun 2017 14:35:31 GMT
I0622 17:35:01.072994   15981 request.go:991] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"private-reg","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/private-reg","uid":"f4340638-5754-11e7-978a-08002773375c","resourceVersion":"3070","creationTimestamp":"2017-06-22T14:13:24Z"},"spec":{"volumes":[{"name":"default-token-bhvgz","secret":{"secretName":"default-token-bhvgz","defaultMode":420}}],"containers":[{"name":"private-reg-container","image":"ha/prod:latest","resources":{},"volumeMounts":[{"name":"default-token-bhvgz","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"minikube","securityContext":{},"imagePullSecrets":[{"name":"regsecret"}],"schedulerName":"default-scheduler"},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2017-06-22T14:13:24Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2017-06-22T14:13:24Z","reason":"ContainersNotReady","message":"containers with unready status: [private-reg-container]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2017-06-22T14:13:24Z"}],"hostIP":"192.168.99.100","podIP":"172.17.0.5","startTime":"2017-06-22T14:13:24Z","containerStatuses":[{"name":"private-reg-container","state":{"waiting":{"reason":"CrashLoopBackOff","message":"Back-off 5m0s restarting failed container=private-reg-container pod=private-reg_default(f4340638-5754-11e7-978a-08002773375c)"}},"lastState":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":null,"finishedAt":"2017-06-22T14:30:36Z","containerID":"docker://a4cb436a79b0b21bb385e544d424b2444a80ca66160ef21af30ab69ed2e23b32"}},"ready":false,"restartCount":8,"image":"ha/prod:latest","imageID":"docker://sha256:7335859e2071af518bcd0e2f373f57c1da643bb37c7e6bbc125d171ff98f71c0","containerID":"docker://a4cb436a79b0b21bb385e544d424b2444a80ca66160ef21af30ab69ed2e23b32"}],"qosClass":"BestEffort"}}
I0622 17:35:01.074108   15981 round_trippers.go:395] GET https://192.168.99.100:8443/api/v1/namespaces/default/pods/private-reg/log
I0622 17:35:01.074126   15981 round_trippers.go:402] Request Headers:
I0622 17:35:01.074132   15981 round_trippers.go:405]     Accept: application/json, */*
I0622 17:35:01.074137   15981 round_trippers.go:405]     User-Agent: kubectl/v1.6.6 (linux/amd64) kubernetes/7fa1c17
I0622 17:35:01.079257   15981 round_trippers.go:420] Response Status: 200 OK in 5 milliseconds
I0622 17:35:01.079289   15981 round_trippers.go:423] Response Headers:
I0622 17:35:01.079299   15981 round_trippers.go:426]     Content-Type: text/plain
I0622 17:35:01.079307   15981 round_trippers.go:426]     Content-Length: 0
I0622 17:35:01.079315   15981 round_trippers.go:426]     Date: Thu, 22 Jun 2017 14:35:31 GMT

我该如何调试此问题?

  • 更新 输出:

  • kubectl --v = 8 log ps-agent-2028336249-3pk43 --namespace = default -p

    I0625 11:30:01.569903 13420 round_trippers.go:395] GET I0625 11:30:01.569920 13420 round_trippers.go:402]请求标题: I0625 11:30:01.569927 13420 round_trippers.go:405] User-Agent:kubectl / v1.6.6(linux / amd64)kubernetes / 7fa1c17 I0625 11:30:01.569934 13420 round_trippers.go:405]接受:application / json, / I0625 11:30:01.599026 13420 round_trippers.go:420]响应状态:200毫秒,29毫秒 I0625 11:30:01.599048 13420 round_trippers.go:423]回复标题: I0625 11:30:01.599056 13420 round_trippers.go:426]日期:2017年6月25日星期日08:30:01 GMT I0625 11:30:01.599062 13420 round_trippers.go:426]内容类型:application / json I0625 11:30:01.599069 13420 round_trippers.go:426]内容长度:2794 I0625 11:30:01.599264 13420 request.go:991]回复正文:{&#34; kind&#34;:&#34; Pod&#34;,&#34; apiVersion&#34;:&#34; v1&# 34;,&#34;元数据&#34; {&#34;名称&#34;:&#34; PS-剂2028336249-3pk43&#34;&#34; generateName&#34;:&#34; PS-剂-2028336249 - &#34;&#34;命名空间&#34;:&#34;默认&#34;&#34; selfLink&#34;:&#34; / API / V1 /命名空间/默认/荚/ PS-剂2028336249-3pk43&#34;&#34; UID&#34;:&#34; 87c69072-597e-11e7-83cd-08002773375c&#34;&#34; resourceVersion&#34;:& #34; 14354&#34;&#34; creationTimestamp&#34;:&#34; 2017-06-25T08:16:03Z&#34;&#34;标签&#34; {&#34;料盒模板哈希&#34;:&#34; 2028336249&#34;&#34;运行&#34;:&#34; PS-剂&#34;}&#34;注释&#34; {&#34 ; kubernetes.io/created-by":&#34; {\&#34;样\&#34;:\&#34; SerializedReference \&#34; \&#34; apiVersion \&# 34;:\&#34; V1 \&#34; \&#34;参考\&#34;:{\&#34;样\&#34;:\&#34; ReplicaSet \&#34 ; \&#34;命名空间\&#34;:\&#34;默认\&#34; \&#34;名称\&#34;:\&#34; PS-剂-2028336249 \& #34; \&#34; UID \&#34;:\&#34; 87c577b5-597e-11e7-83cd-08002773375 Ç\&#34; \&#34; apiVersion \&#34;:\&#34;扩展\&#34; \&#34; resourceVersion \&#34;:\&#34; 13446 \ &#34;}} \ n&#34;}&#34; ownerReferences&#34;:[{&#34; apiVersion&#34;:&#34;扩展/ v1beta1&#34;&#34;种类&# 34;:&#34; ReplicaSet&#34;&#34;名称&#34;:&#34; PS-剂-2028336249&#34;&#34; UID&#34;:&#34; 87c577b5-597e -11e7-83cd-08002773375c&#34;&#34;控制器&#34;:真,&#34; blockOwnerDeletion&#34;:真}]},&#34;规格&#34; {&#34;卷& #34;:#34:[{&#34;名称&#34;::&#34;默认-令牌bhvgz&#34;&#34;秘密&#34; {&#34; secretName&#34 ;默认-令牌bhvgz&#34;&#34; defaultMode&#34;:420}}],&#34;容器&#34;:[{&#34;名称&#34;:&#34; PS-剂&#34;&#34;图像&#34;:&#34;公顷/ PROD:PS-剂最新&#34;&#34;资源&#34;:{},&#34; volumeMounts&#34 ;:[{&#34;名称&#34;:&#34;默认-令牌bhvgz&#34;&#34; readOnly的&#34;:真,&#34;装载路径&#34;:&#34; /var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy" ;: &#34;文件&#34;&#34; imagePu llPolicy&#34;:&#34; IfNotPresent&#34;}],&#34; restartPolicy&#34;:&#34;总是&#34;&#34; terminationGracePeriodSeconds&#34;:30,&#34; dnsPolicy& #34;:&#34; ClusterFirst&#34;&#34; serviceAccountName&#34;:&#34;默认&#34;&#34; serviceAccount&#34;:&#34;默认&#34;,& #34; nodeName的&#34;:&#34; minikube&#34;&#34; SecurityContext的&#34;:{},&#34; schedulerName&#34;:&#34;默认调度&#34;}, &#34;状态&#34; {&#34;相&#34;:&#34;运行&#34;&#34;条件&#34;:[{&#34;类型&#34;:&# 34;初始化&#34;&#34;状态&#34;:&#34;真&#34;&#34; lastProbeTime&#34;:空,&#34; lastTransitionTime&#34;:&#34; 2017年-06-25T08:16:03Z&#34;},{&#34;类型&#34;:&#34;就绪&#34;&#34;状态&#34;:&#34;假&#34 ;, &#34; lastProbeTime&#34;:空,&#34; lastTransitionTime&#34;:&#34; 2017-06-25T08:16:03Z&#34;&#34;理由&#34;:&#34; ContainersNotReady&#34;,&#34; message&#34;:&#34;状态为未来的容器:[ps-agent]&#34;},{&#34; type&#34;:&#34; PodScheduled&# 34;,&#34;状态&#34;:&#34;真&#34;&#34; lastProbeTime&#34;:空,&#34; lastTransitio n时间&#34;:&#34; 2017-06-25T08:16:03Z&#34;}],&#34;主机IP&#34;:&#34; 192.168.99.100&#34;&#34;&podIP #34;:&#34; 172.17.0.5&#34;&#34; STARTTIME&#34;:&#34; 2017-06-25T08:16:03Z&#34;&#34; containerStatuses&#34; :[{&#34;名称&#34;:&#34; PS-剂&#34;&#34;状态&#34; {&#34;等待&#34; {&#34;理由&#34 ;:&#34; CrashLoopBackOff&#34;,&#34; message&#34;:&#34; Back-off 5m0s重启失败的容器= ps-agent pod = ps-agent-2028336249-3pk43_default(87c69072-597e-11e7 -83cd-08002773375c)&#34;}},&#34; lastState&#34; {&#34;终止&#34; {&#34; EXITCODE&#34;:0,&#34;理由&#34 ;:&#34;完成&#34;&#34; startedAt&#34;:空,&#34; finishedAt&#34;:&#34; 2017-06-25T08:27:17Z&#34;&# 34;数据筒&#34;:&#34;搬运工:// 1aa9dfbfeb80042c6f4c8d04cabb3306ac1cd52963568e621019e2f1f0ee081b&#34;}},&#34;准备&#34;:假,&#34; RESTARTCOUNT&#34;:7,&#34;图像&# 34;:&#34;公顷/ PROD:PS-剂最新&#34;&#34;图像标识&#34;:&#34;搬运工:// SHA256:eb5307c4366fc129d022703625a5f30ff175b5e1a24dbe39fd4c32e726a0ee7b&#34;&#34;数据筒&# 34;:&#34;搬运工:// 1aa9dfbfeb80042c6f4c8d04cabb3306ac1cd52963568e621019e2f1f0ee081b&#34;}],&#34; qosClass&#34;:&#34; BestEffort&#34;}} I0625 11:30:01.600727 13420 round_trippers.go:395] GET https://192.168.99.100:8443/api/v1/namespaces/default/pods/ps-agent-2028336249-3pk43/log?previous=true I0625 11:30:01.600747 13420 round_trippers.go:402]请求标题: I0625 11:30:01.600757 13420 round_trippers.go:405]接受:application / json, / I0625 11:30:01.600766 13420 round_trippers.go:405] User-Agent:kubectl / v1.6.6(linux / amd64)kubernetes / 7fa1c17 I0625 11:30:01.632473 13420 round_trippers.go:420]响应状态:31毫秒内200 OK I0625 11:30:01.632545 13420 round_trippers.go:423]回复标题: I0625 11:30:01.632569 13420 round_trippers.go:426]日期:2017年6月25日星期日08:30:01 GMT I0625 11:30:01.632592 13420 round_trippers.go:426]内容类型:text / plain I0625 11:30:01.632615 13420 round_trippers.go:426]内容长度:0

6 个答案:

答案 0 :(得分:16)

由于#dock;#34; start&#34;过程结束。我添加了一个永远运行的命令,它工作正常。这个问题提到了here

答案 1 :(得分:2)

由于POD初始化后出现问题,无法启动Pod。

检查并使用命令获取Pod的Docker容器

docker ps -a | grep private-reg

输出将是具有ID的docker容器的信息。

请参阅docker日志:

docker logs -f <container id>

答案 2 :(得分:1)

我遇到了同样的错误。

NAME         READY   STATUS             RESTARTS   AGE
pod/webapp   0/1     CrashLoopBackOff   5          47h

我的问题是我试图使用相同的元数据名称运行两个不同的pod。

种类:豆荚 元数据:   名称:webapp   标签:...

要查找所有Pod的名称,请运行: kubectl获得豆荚

NAME         READY   STATUS    RESTARTS   AGE
webapp       1/1     Running   15         47h

然后我更改了有冲突的pod名称,一切正常。

NAME                 READY   STATUS    RESTARTS   AGE
webapp               1/1     Running   17         2d
webapp-release-0-5   1/1     Running   0          13m

答案 3 :(得分:1)

当我调试获取Pod和Pod的日志时,我遇到了类似的问题“ CrashLoopBackOff”。发现我的命令参数错误

答案 4 :(得分:1)

问题在此处描述并解决 https://docs.pivotal.io/tkgi/1-10/velero-install.html。需要正确设置主机路径。

答案 5 :(得分:0)

我也有类似情况。我发现我的配置映射之一被复制了。我有两个同一个命名空间的配置映射。一个具有正确的名称空间引用,另一个则指向错误的名称空间。

我删除并使用正确的文件(或固定文件)重新创建了configmap。我只使用一个,这似乎使特定群集更快乐。

因此,我将检查文件中是否有可能引起冲突的错别字或重复项。