水平Pod自动缩放无法读取指标

时间:2019-01-18 06:15:32

标签: kubernetes apache-kafka kubernetes-helm

我正在使用here中的Kafka Helm图表。 我在尝试使用水平Pod自动缩放器。

我在模板文件夹中添加了一个hpa.yaml文件,如下所示。

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: kafka-hpa
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: {{ include "kafka.fullname" . }}
minReplicas: {{ .Values.replicas }}
maxReplicas: 5
metrics:
- type: Resource
  resource:
    name: cpu
    targetAverageUtilization: 50
- type: Resource
  resource:
    name: memory
    targetAverageValue: 8000Mi

我还尝试了使用 kind:StatefulSet 的上述YAML,但是仍然存在相同的问题。

我的意图是最初拥有3个Kafka Pod,并根据上述CPU和内存targetValues将其扩展到5个。

但是,根据我的理解,hpa已部署,但是由于当前用法显示未知,因此无法读取指标。

NAME        REFERENCE                          TARGETS                          MINPODS   MAXPODS   REPLICAS   AGE
kafka-hpa   Deployment/whopping-walrus-kafka   <unknown>/8000Mi, <unknown>/50%   3         5         0          1h . 

我是掌舵人和Kubernetes的新手,所以我认为我的理解可能存在一些问题。

我还部署了metrics-server。

$ kubectl get deployments
NAME                             DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
metrics-server                   1         1         1            1           1d
whopping-walrus-kafka-exporter   1         1         1            1           1h

豆荚输出

$ kubectl get pods
NAME                                              READY     STATUS    RESTARTS   AGE
metrics-server-55cbf87bbb-vm2v5                   1/1       Running   0          15m
whopping-walrus-kafka-0                           1/1       Running   1          1h
whopping-walrus-kafka-1                           1/1       Running   0          1h
whopping-walrus-kafka-2                           1/1       Running   0          1h
whopping-walrus-kafka-exporter-5c66b5b4f9-mv5kv   1/1       Running   1          1h
whopping-walrus-zookeeper-0                       1/1       Running   0          1h

我希望 whopping-walrus-kafka 窗格在加载时最多可扩展到5个,但是没有对应的部署。

StatefulSet输出

$ kubectl get statefulset
NAME                        DESIRED   CURRENT   AGE
original-bobcat-kafka       3         2         2m
original-bobcat-zookeeper   1         1         2m

hpa.yaml中的kind为 StatefulSet 时,描述hpa的输出。

$ kubectl describe hpa
Name:                                                  kafka-hpa
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Fri, 18 Jan 2019 12:13:59 +0530
Reference:                                             StatefulSet/original-bobcat-kafka
Metrics:                                               ( current / target )
  resource memory on pods:                             <unknown> / 8000Mi
  resource cpu on pods  (as a percentage of request):  <unknown> / 5%
Min replicas:                                          3
Max replicas:                                          5
Conditions:
  Type         Status  Reason          Message
  ----         ------  ------          -------
  AbleToScale  False   FailedGetScale  the HPA controller was unable to get the target's current scale: no matches for kind "StatefulSet" in group "extensions"
Events:
  Type     Reason          Age                From                       Message
  ----     ------          ----               ----                       -------
  Warning  FailedGetScale  15s (x17 over 8m)  horizontal-pod-autoscaler  no matches for kind "StatefulSet" in group "extensions"

hpa.yaml中的kind为部署时,描述hpa的输出。

$ kubectl describe hpa
Name:                                                  kafka-hpa
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Fri, 18 Jan 2019 12:30:07 +0530
Reference:                                             Deployment/good-elephant-kafka
Metrics:                                               ( current / target )
  resource memory on pods:                             <unknown> / 8000Mi
  resource cpu on pods  (as a percentage of request):  <unknown> / 5%
Min replicas:                                          3
Max replicas:                                          5
Conditions:
  Type         Status  Reason          Message
  ----         ------  ------          -------
  AbleToScale  False   FailedGetScale  the HPA controller was unable to get the target's current scale: could not fetch the scale for deployments.extensions good-elephant-kafka: deployments/scale.extensions "good-elephant-kafka" not found
Events:
  Type     Reason          Age   From                       Message
  ----     ------          ----  ----                       -------
  Warning  FailedGetScale  9s    horizontal-pod-autoscaler  could not fetch the scale for deployments.extensions good-elephant-kafka: deployments/scale.extensions "good-elephant-kafka" not found

指标服务器容器的输出

$ kubectl describe pods metrics-server-55cbf87bbb-vm2v5
Name:           metrics-server-55cbf87bbb-vm2v5
Namespace:      default
Node:           docker-for-desktop/192.168.65.3
Start Time:     Fri, 18 Jan 2019 11:26:33 +0530
Labels:         app=metrics-server
            pod-template-hash=1176943666
            release=metrics-server
Annotations:    <none>
Status:         Running
IP:             10.1.0.119
Controlled By:  ReplicaSet/metrics-server-55cbf87bbb
Containers:
  metrics-server:
    Container ID:  docker://ee4b3d9ed1b15c2c8783345b0ffbbc565ad25f1493dec0148f245c9581443631
    Image:         gcr.io/google_containers/metrics-server-amd64:v0.3.1
    Image ID:      docker-pullable://gcr.io/google_containers/metrics-server-amd64@sha256:78938f933822856f443e6827fe5b37d6cc2f74ae888ac8b33d06fdbe5f8c658b
    Port:          <none>
    Host Port:     <none>
    Command:
      /metrics-server
      --kubelet-insecure-tls
      --kubelet-preferred-address-types=InternalIP
      --logtostderr
    State:          Running
  Started:      Fri, 18 Jan 2019 11:26:35 +0530
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
  /var/run/secrets/kubernetes.io/serviceaccount from metrics-server-token-d2g7b (ro)
Conditions:
  Type           Status
  Initialized    True 
  Ready          True 
  PodScheduled   True 
Volumes:
  metrics-server-token-d2g7b:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  metrics-server-token-d2g7b
    Optional:    false
    QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
             node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

如果我在某个地方出错,请大家也清除我的理解。

在此方面的帮助将不胜感激。

3 个答案:

答案 0 :(得分:0)

您需要在metrics-server部署文件中添加以下命令:

containers:
   - command:
     - /metrics-server
     - --metric-resolution=30s
     - --kubelet-insecure-tls
     - --kubelet-preferred-address-types=InternalIP
     name: metrics-server

我认为metrics-server无法找到带有InternalIP的kubelet,因此出现了问题。有关更多信息,请检查我下面的答案以逐步设置HPA。

How to Enable KubeAPI server for HPA Autoscaling Metrics

答案 1 :(得分:0)

我执行了一些操作,类似于上面@PrafullLadha提到的操作。

修改了指标服务器部署文件并添加了以下代码:

containers:
 - command:
  - /metrics-server
  - --metric-resolution=30s
  - --kubelet-insecure-tls 
  - --kubelet-preferred-address-types=InternalIP`

此外,取消注释statefulset.yaml文件中的以下部分

resources: requests: cpu: 200m memory: 256Mi

在上面效果很好。

答案 2 :(得分:0)

如果您的部署到目前为止没有节点,并且群集中可用的节点不满足您的资源要求,那么也会发生此错误。在这种情况下,显然没有可用的指标。​​