Question

我已根据Running Kubernetes Locally via Docker指南在Ubuntu（可靠）上设置了Kubernetes集群，部署了DNS并使用InfluxDB后端和Grafana UI运行Heapster。

除了Grafana之外，一切似乎都顺利进行，Grafana没有显示任何图表，但在图表中显示了No datapoints消息：Screenshot

检查Docker容器日志后，我发现Heapster无法访问kubelet API（？），因此没有指标持续存在于InfluxDB中：

user@host:~$ docker logs e490a3ac10a8
I0701 07:07:30.829745       1 heapster.go:65] /heapster --source=kubernetes:https://kubernetes.default --sink=influxdb:http://monitoring-influxdb:8086
I0701 07:07:30.830082       1 heapster.go:66] Heapster version 1.2.0-beta.0
I0701 07:07:30.830809       1 configs.go:60] Using Kubernetes client with master "https://kubernetes.default" and version v1
I0701 07:07:30.831284       1 configs.go:61] Using kubelet port 10255
E0701 07:09:38.196674       1 influxdb.go:209] issues while creating an InfluxDB sink: failed to ping InfluxDB server at "monitoring-influxdb:8086" - Get http://monitoring-influxdb:8086/ping: dial tcp 10.0.0.223:8086: getsockopt: connection timed out, will retry on use
I0701 07:09:38.196919       1 influxdb.go:223] created influxdb sink with options: host:monitoring-influxdb:8086 user:root db:k8s
I0701 07:09:38.197048       1 heapster.go:92] Starting with InfluxDB Sink
I0701 07:09:38.197154       1 heapster.go:92] Starting with Metric Sink
I0701 07:09:38.228046       1 heapster.go:171] Starting heapster on port 8082
I0701 07:10:05.000370       1 manager.go:79] Scraping metrics start: 2016-07-01 07:09:00 +0000 UTC, end: 2016-07-01 07:10:00 +0000 UTC
E0701 07:10:05.008785       1 kubelet.go:230] error while getting containers from Kubelet: failed to get all container stats from Kubelet URL "http://127.0.0.1:10255/stats/container/": Post http://127.0.0.1:10255/stats/container/: dial tcp 127.0.0.1:10255: getsockopt: connection refused
I0701 07:10:05.009119       1 manager.go:152] ScrapeMetrics: time: 8.013178ms size: 0
I0701 07:11:05.001185       1 manager.go:79] Scraping metrics start: 2016-07-01 07:10:00 +0000 UTC, end: 2016-07-01 07:11:00 +0000 UTC
E0701 07:11:05.007130       1 kubelet.go:230] error while getting containers from Kubelet: failed to get all container stats from Kubelet URL "http://127.0.0.1:10255/stats/container/": Post http://127.0.0.1:10255/stats/container/: dial tcp 127.0.0.1:10255: getsockopt: connection refused
I0701 07:11:05.007686       1 manager.go:152] ScrapeMetrics: time: 5.945236ms size: 0
W0701 07:11:25.010298       1 manager.go:119] Failed to push data to sink: InfluxDB Sink
I0701 07:12:05.000420       1 manager.go:79] Scraping metrics start: 2016-07-01 07:11:00 +0000 UTC, end: 2016-07-01 07:12:00 +0000 UTC
E0701 07:12:05.002413       1 kubelet.go:230] error while getting containers from Kubelet: failed to get all container stats from Kubelet URL "http://127.0.0.1:10255/stats/container/": Post http://127.0.0.1:10255/stats/container/: dial tcp 127.0.0.1:10255: getsockopt: connection refused
I0701 07:12:05.002467       1 manager.go:152] ScrapeMetrics: time: 1.93825ms size: 0
E0701 07:12:12.309151       1 influxdb.go:150] Failed to create infuxdb: failed to ping InfluxDB server at "monitoring-influxdb:8086" - Get http://monitoring-influxdb:8086/ping: dial tcp 10.0.0.223:8086: getsockopt: connection timed out
I0701 07:12:12.351348       1 influxdb.go:201] Created database "k8s" on influxDB server at "monitoring-influxdb:8086"
I0701 07:13:05.001052       1 manager.go:79] Scraping metrics start: 2016-07-01 07:12:00 +0000 UTC, end: 2016-07-01 07:13:00 +0000 UTC
E0701 07:13:05.015947       1 kubelet.go:230] error while getting containers from Kubelet: failed to get all container stats from Kubelet URL "http://127.0.0.1:10255/stats/container/": Post http://127.0.0.1:10255/stats/container/: dial tcp 127.0.0.1:10255: getsockopt: connection refused
...

我在GitHub上发现了一些问题，描述了类似的问题让我明白Heapster不会访问kubelet（通过节点的环回），而是自己（通过容器的环回）。但是，我无法重现他们的解决方案：

github.com/kubernetes/heapster/issues/1183

您应该为Heapster pod使用主机网络，或者以节点具有常规名称而不是127.0.0.1的方式配置群集。当前的问题是节点名称被解析为Heapster localhost。如有更多问题，请重新开启。

- @ piosz

如何启用＆＃34;主机网络＆＃34;我的Heapster吊舱？
如何将我的群集/节点配置为使用常规名称而不是127.0.0.1？

github.com/kubernetes/heapster/issues/744

通过在hyperkube中使用更好的选项进行修复，感谢您的帮助！

- @ ddispaltro

有没有办法通过在docker run中添加/修改kubelet的选项标记来解决此问题？
我尝试设置--hostname-override=<host's eth0 IP>和--address=127.0.0.1（如此GitHub问题的最后一个答案所示），但Heapster的容器日志则说明：

{ {1}}

命名空间问题

这个问题可能是因为我在I0701 08:23:05.000566 1 manager.go:79] Scraping metrics start: 2016-07-01 08:22:00 +0000 UTC, end: 2016-07-01 08:23:00 +0000 UTC E0701 08:23:05.000962 1 kubelet.go:279] Node 127.0.0.1 is not ready E0701 08:23:05.003018 1 kubelet.go:230] error while getting containers from Kubelet: failed to get all container stats from Kubelet URL "http://<host's eth0 IP>:10255/stats/container/": Post http://<host's eth0 IP>/stats/container/: dial tcp <host's eth0 IP>:10255: getsockopt: connection refused命名空间中运行Kubernetes API而在default中运行了Heapster吗？

kube-system

操作系统：Ubuntu 14.04.4 LTS（可靠）| Kubernetes：v1.2.5 | Docker：v1.11.2

Answer 1

Heapster已经获得了Kubernetes的节点列表，现在正在尝试从每个节点上的kublete进程中提取统计数据（其中有一个内置的cAdvisor收集节点上的统计信息）。在这种情况下，只有一个节点，并且通过127.0.0.1知道kubernetes。而且存在问题。 Heapster容器正试图到达127.0.0.1的节点，这本身就是当然，并且找不到在Heapster容器内查询的kublete进程。

解决此问题需要做两件事。

我们需要通过除环回网络地址127.0.0.1之外的其他内容来引用kublete worker节点（运行kubernetes的主机）
kublete进程需要接受来自新网络接口/地址的流量

假设您正在使用本地安装指南并使用

hack/local-up-cluster.sh

更改引用kublete的主机名非常简单。您可以采取更详细的方法，但将此设置为eth0 ip对我来说很好（ifconfig eth0）。缺点是你需要一个eth0接口，这取决于DHCP，所以你的里程可能会有多么方便。

export HOSTNAME_OVERRIDE=10.0.2.15

让kublete进程接受来自任何网络接口的流量同样简单。

export KUBELET_HOST=0.0.0.0

Answer 2

为您的heapster配置提供以下参数以解决该问题。

-source = kubernetes：https://kubernetes.default:443?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true

Heapster无法在Kubernetes集群上从Kubelet获取容器统计信息

2 个答案: