kubernetes如何使用HPA与公制服务器自动缩放?

时间:2019-05-07 01:18:19

标签: kubernetes

我非常感兴趣在ubuntu安装中测试kubernete自动缩放解决方案。我已经在带有堆的minikube中使用了它,但是由于它已被弃用,因此我尝试使用度量服务器。 现在在我的ubuntu中,我安装了如下的metrics-server:

kube-system      kube-apiserver-kmaster                  1/1     Running   1          11d
kube-system      kube-controller-manager-kmaster         1/1     Running   1          11d
kube-system      kube-proxy-47k6b                        1/1     Running   0          11d
kube-system      kube-proxy-q8zdw                        1/1     Running   1          11d
kube-system      kube-scheduler-kmaster                  1/1     Running   1          11d
kube-system      kubernetes-dashboard-5f7b999d65-6wl6k   1/1     Running   1          11d
kube-system      metrics-server-548456b4cd-wxc9b         1/1     Running   0          3d18h
metallb-system   controller-cd8657667-ckpn6              1/1     Running   0          8d
metallb-system   speaker-m9599   

但是当我检查HPA时,总是看到以下内容:

kubectl获得hpa

NAME         REFERENCE               TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
api-server   Deployment/api-server   <unknown>/50%   1         10        3          3d19h
ngsc         Deployment/ngsc         <unknown>/50%   1         10        3          3d19h

似乎不使用度量​​标准服务来计算使用量。

我曾到过kubernete doc站点,但真的不知道如何配置度量服务器的利用率,以便kubernete自动缩放。有人可以帮我解决这个问题吗?

谢谢。

更多信息,我描述了自动缩放:

                        api-server
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Fri, 03 May 2019 05:49:07 +0000
Reference:                                             Deployment/api-server
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  <unknown> / 50%
Min replicas:                                          1
Max replicas:                                          10
Deployment pods:                                       3 current / 0 desired
Conditions:
  Type           Status  Reason                   Message
  ----           ------  ------                   -------
  AbleToScale    True    SucceededGetScale        the HPA controller was able to get the target's current scale
  ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count: unable to get metrics for resource cpu: no metrics returned from resource metrics API
Events:
  Type     Reason                   Age                        From                       Message
  ----     ------                   ----                       ----                       -------
  Warning  FailedGetResourceMetric  4m48s (x22069 over 3d20h)  horizontal-pod-autoscaler  unable to get metrics for resource cpu: no metrics returned from resource metrics API

描述部署:

Pod Template:
  Labels:  app=api-server
  Containers:
   api-server:
    Image:      xxxxxx
    Port:       <none>
    Host Port:  <none>
    Limits:
      cpu:  500m
    Requests:
      cpu:        200m
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>

这意味着部署具有资源配置。但是hpa仍然显示未知

添加内存,现在描述的是:

 Limits:
      cpu:     500m
      memory:  1Gi
    Requests:
      cpu:        500m
      memory:     512Mi

但是kubectl get hpa仍然未知。

检查度量服务器的日志:

 1 manager.go:111] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:kmaster: unable to fetch metrics from Kubelet kmaster (kmaster): Get https://kmaster:10250/stats/summary/: dial tcp: lookup kmaster on 10.96.0.10:53: no such host, unable to fully scrape metrics from source kubelet_summary:knode: unable to fetch metrics from Kubelet knode (knode): Get https://knode:10250/stats/summary/: dial tcp: lookup knode on 10.96.0.10:53: no such host]
E0507 05:20:23.797590       1 reststorage.go:148] unable to fetch pod metrics for pod default/api-server-777b78ccf5-mlt94: no metrics known for pod
E0507 05:20:23.797614       1 reststorage.go:148] unable to fetch pod metrics for pod default/api-server-777b78ccf5-r66bw: no metrics known for pod

以及curl -k https://knode:10250/stats/summary/时 我知道这个错误: 未经授权

3 个答案:

答案 0 :(得分:0)

有时会发生HPA没有显示该值的情况,请确保度量服务器的Pod在命名空间kube-system中运行。

有时对我来说,当网站上出现流量时,它开始显示HPA中的值。

答案 1 :(得分:0)

这意味着Pod没有分配任何CPU资源。没有分配资源,HPA无法做出扩展决策。尝试像这样向吊舱添加一些资源:

spec:
  containers:
  - resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

答案 2 :(得分:0)

根据您提供的信息。

有了pod metrics-server-548456b4cd-wxc9b,就意味着已启用metric-server。另外,由于您有3个副本,因此我认为此编号是在“部署清单”中提供的。

由于以下原因,HPA可能无法扩展您的部署:

1)资源不足

$ kubectl describe node
...
 Namespace                  Name                                 CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                  ----                                 ------------  ----------  ---------------  -------------  ---
  default                    nginx-deployment-5ffb677f99-k5mdj    200m (10%)    500m (25%)  0 (0%)           0 (0%)         6m55s
  default                    nginx-deployment-5ffb677f99-n7t7n    200m (10%)    500m (25%)  0 (0%)           0 (0%)         6m55s
  default                    nginx-deployment-5ffb677f99-pw2g7    200m (10%)    500m (25%)  0 (0%)           0 (0%)         6m55s
  kube-system                etcd-minikube                        0 (0%)        0 (0%)      0 (0%)           0 (0%)         152m
  kube-system                kube-addon-manager-minikube          5m (0%)       0 (0%)      50Mi (0%)        0 (0%)         152m
  kube-system                kube-apiserver-minikube              250m (12%)    0 (0%)      0 (0%)           0 (0%)         152m
  kube-system                kube-controller-manager-minikube     200m (10%)    0 (0%)      0 (0%)           0 (0%)         152m
  kube-system                kube-dns-6bfbdd666c-l74lx            260m (13%)    0 (0%)      110Mi (1%)       170Mi (2%)     32m
  kube-system                kube-proxy-dnh4m                     0 (0%)        0 (0%)      0 (0%)           0 (0%)         153m
  kube-system                kube-scheduler-minikube              100m (5%)     0 (0%)      0 (0%)           0 (0%)         152m
  kube-system                metrics-server-77fddcc57b-mjlf5      0 (0%)        0 (0%)      0 (0%)           0 (0%)         147m
  kube-system                storage-provisioner                  0 (0%)        0 (0%)      0 (0%)           0 (0%)         153m
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests     Limits
  --------           --------     ------
  cpu                1415m (70%)  1500m (75%)
  memory             160Mi (2%)   170Mi (2%)
  ephemeral-storage  0 (0%)       0 (0%)

如示例所示,minikube资源和3个具有nginx的容器已经请求了70%的CPU。 在清单中,每个容器都将请求cpu: 200m,因此此部署只能再创建2个容器。由于缺少CPU资源,其他Pod将处于Pending状态。

2)缺少CPU负载

类似the HPA was unable to compute the replica count: unable to get metrics for resource cpu: no metrics returned from resource metrics API的错误消息表示,度量服务器未收到任何度量,因此pod不会产生任何负载。

我假设您使用命令扩展部署

$ kubectl autoscale deployment api-server --cpu-percent=50 --min=1 --max=10
...
Conditions:
  Type           Status  Reason                   Message
  ----           ------  ------                   -------
  AbleToScale    True    SucceededGetScale        the HPA controller was able to get the target's current scale
  ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count: unable to get metrics for resource cpu: no metrics returned from resource metrics API
Events:
  Type     Reason                        Age   From                       Message
  ----     ------                        ----  ----                       -------
  Warning  FailedGetResourceMetric       9s    horizontal-pod-autoscaler  unable to get metrics for resource cpu: no metrics returned from resource metrics API
  Warning  FailedComputeMetricsReplicas  9s    horizontal-pod-autoscaler  failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

尝试通过输入部署窗格之一来产生一些CPU负载

$ kubectl exec -ti <yourPodName> sh

$ while true; do echo 'IncreaseLoad'; done
IncreaseLoad
IncreaseLoad
IncreaseLoad
...

您也可以使用stress命令。

一段时间后,HPA应该获取指标并将其从更改为正确的值。

Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:
  Type     Reason                        Age                From                       Message
  ----     ------                        ----               ----                       -------
  Warning  FailedGetResourceMetric       14m (x6 over 16m)  horizontal-pod-autoscaler  unable to get metrics for resource cpu: no metrics returned from resource metrics API
  Warning  FailedComputeMetricsReplicas  14m (x6 over 16m)  horizontal-pod-autoscaler  failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
  Normal   SuccessfulRescale             6m54s              horizontal-pod-autoscaler  New size: 2; reason: All metrics below target
  Normal   SuccessfulRescale             50s                horizontal-pod-autoscaler  New size: 4; reason: cpu resource utilization (percentage of request) above target

如果这没有帮助,请提供您的HPA和部署清单。