如何让kubelet使用HTTPS与apiserver通信？ v0.19

时间：2015-06-30 12:00:04

标签： kubernetes

我在主节点（core01）上部署了apiserver，其中包含以下conf：

core01> /opt/bin/kube-apiserver \
  --insecure_bind_address=127.0.0.1 \
  --insecure_port=8080 \
  --kubelet_port=10250 \
  --etcd_servers=http://core01:2379,http://core02:2379,http://core03:2379 \
  --service-cluster-ip-range=10.1.0.0/16 \
  --allow_privileged=false \
  --logtostderr=true \
  --v=5 \
  --tls-cert-file="/var/run/kubernetes/apiserver_36kr.pem" \
  --tls-private-key-file="/var/run/kubernetes/apiserver_36kr.key" \
  --client-ca-file="/var/run/kubernetes/cacert.pem" \
  --kubelet-certificate-authority="/var/run/kubernetes/cacert.pem" \
  --kubelet-client-certificate="/var/run/kubernetes/kubelet_36kr.pem" \
  --kubelet-client-key="/var/run/kubernetes/kubelet_36kr.key"

在minion节点（core02）上，我可以从HTTPS调用api：

core02> curl https://core01:6443/api/v1/nodes --cert /var/run/kubernetes/kubelet_36kr.pem --key /var/run/kubernetes/kubelet_36kr.key
> GET /api/v1/nodes HTTP/1.1
> Host: core01:6443
> User-Agent: curl/7.42.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sat, 27 Jun 2015 15:33:50 GMT
< Content-Length: 1577
< 
{
  "kind": "NodeList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/nodes",
    "resourceVersion": "510078"
  }, ....

然而，我不能在这个小兵上开始kubelet。它总是抱怨没有凭据。

我怎样才能让它发挥作用？主人是否有任何文档＆lt; - ＆gt; minion通信认证？你能告诉我最好的做法吗？

仅供参考，命令如下：

core02> /opt/bin/kubelet \
  --logtostderr=true \
  --v=0 \
  --api_servers=https://core01:6443 \
  --address=127.0.0.1 \
  --port=10250 \
  --allow-privileged=false \
  --tls-cert-file="/var/run/kubernetes/kubelet_36kr.pem" \
  --tls-private-key-file="/var/run/kubernetes/kubelet_36kr.key"

kubelet日志如下：

W0627 23:34:03.646311    3004 server.go:460] Could not load kubeconfig file /var/lib/kubelet/kubeconfig: stat /var/lib/kubelet/kubeconfig: no such file or directory. Trying auth path instead.
W0627 23:34:03.646520    3004 server.go:422] Could not load kubernetes auth path /var/lib/kubelet/kubernetes_auth: stat /var/lib/kubelet/kubernetes_auth: no such file or directory. Continuing with defaults.
I0627 23:34:03.646710    3004 manager.go:127] cAdvisor running in container: "/system.slice/sshd.service"
I0627 23:34:03.647292    3004 fs.go:93] Filesystem partitions: map[/dev/sda9:{mountpoint:/ major:0 minor:30} /dev/sda4:{mountpoint:/usr major:8 minor:4} /dev/sda6:{mountpoint:/usr/share/oem major:8 minor:6}]
I0627 23:34:03.648234    3004 manager.go:156] Machine: {NumCores:1 CpuFrequency:2399996 MemoryCapacity:1046294528 MachineID:29f94a4fad8b31668bd219ca511bdeb0 SystemUUID:4F4AF929-8BAD-6631-8BD2-19CA511BDEB0 BootID:fa1bea28-675e-4989-ad86-00797721a794 Filesystems:[{Device:/dev/sda9 Capacity:18987593728} {Device:/dev/sda4 Capacity:1031946240} {Device:/dev/sda6 Capacity:113229824}] DiskMap:map[8:0:{Name:sda Major:8 Minor:0 Size:21474836480 Scheduler:cfq} 8:16:{Name:sdb Major:8 Minor:16 Size:1073741824 Scheduler:cfq}] NetworkDevices:[{Name:eth0 MacAddress:52:54:71:f6:fc:b8 Speed:0 Mtu:1500} {Name:flannel0 MacAddress: Speed:10 Mtu:1472}] Topology:[{Id:0 Memory:1046294528 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}]}
I0627 23:34:03.649934    3004 manager.go:163] Version: {KernelVersion:4.0.5 ContainerOsVersion:CoreOS 695.2.0 DockerVersion:1.6.2 CadvisorVersion:0.15.1}
I0627 23:34:03.651758    3004 plugins.go:69] No cloud provider specified.
I0627 23:34:03.651855    3004 docker.go:289] Connecting to docker on unix:///var/run/docker.sock
I0627 23:34:03.652877    3004 server.go:659] Watching apiserver
E0627 23:34:03.748954    3004 reflector.go:136] Failed to list *api.Pod: the server has asked for the client to provide credentials (get pods)
E0627 23:34:03.750157    3004 reflector.go:136] Failed to list *api.Node: the server has asked for the client to provide credentials (get nodes)
E0627 23:34:03.751666    3004 reflector.go:136] Failed to list *api.Service: the server has asked for the client to provide credentials (get services)
I0627 23:34:03.758158    3004 plugins.go:56] Registering credential provider: .dockercfg
I0627 23:34:03.856215    3004 server.go:621] Started kubelet
E0627 23:34:03.858346    3004 kubelet.go:662] Image garbage collection failed: unable to find data for container /
I0627 23:34:03.869739    3004 kubelet.go:682] Running in container "/kubelet"
I0627 23:34:03.869755    3004 server.go:63] Starting to listen on 127.0.0.1:10250
E0627 23:34:03.899877    3004 event.go:185] Server rejected event '&api.Event{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"core02.13eba23275ceda25", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:util.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*util.Time)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"core02", UID:"core02", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"starting", Message:"Starting kubelet.", Source:api.EventSource{Component:"kubelet", Host:"core02"}, FirstTimestamp:util.Time{Time:time.Time{sec:63571016043, nsec:856189989, loc:(*time.Location)(0x1ba6120)}}, LastTimestamp:util.Time{Time:time.Time{sec:63571016043, nsec:856189989, loc:(*time.Location)(0x1ba6120)}}, Count:1}': 'the server has asked for the client to provide credentials (post events)' (will not retry!)
I0627 23:34:04.021297    3004 factory.go:226] System is using systemd
I0627 23:34:04.021790    3004 factory.go:234] Registering Docker factory
I0627 23:34:04.022241    3004 factory.go:89] Registering Raw factory
I0627 23:34:04.144065    3004 manager.go:946] Started watching for new ooms in manager
I0627 23:34:04.144655    3004 oomparser.go:183] oomparser using systemd
I0627 23:34:04.145379    3004 manager.go:243] Starting recovery of all containers
I0627 23:34:04.293020    3004 manager.go:248] Recovery completed
I0627 23:34:04.343829    3004 status_manager.go:56] Starting to sync pod status with apiserver
I0627 23:34:04.343928    3004 kubelet.go:1683] Starting kubelet main sync loop.
E0627 23:34:04.457765    3004 event.go:185] Server rejected event '&api.Event{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"core02.13eba232995c8213", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:util.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*util.Time)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"core02", UID:"core02", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeReady", Message:"Node core02 status is now: NodeReady", Source:api.EventSource{Component:"kubelet", Host:"core02"}, FirstTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:452676115, loc:(*time.Location)(0x1ba6120)}}, LastTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:452676115, loc:(*time.Location)(0x1ba6120)}}, Count:1}': 'the server has asked for the client to provide credentials (post events)' (will not retry!)
E0627 23:34:04.659874    3004 event.go:185] Server rejected event '&api.Event{TypeMeta:api.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:api.ObjectMeta{Name:"core02.13eba232a599cf8c", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:util.Time{Time:time.Time{sec:0, nsec:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*util.Time)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil)}, InvolvedObject:api.ObjectReference{Kind:"Node", Namespace:"", Name:"core02", UID:"core02", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeReady", Message:"Node core02 status is now: NodeReady", Source:api.EventSource{Component:"kubelet", Host:"core02"}, FirstTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:658020236, loc:(*time.Location)(0x1ba6120)}}, LastTimestamp:util.Time{Time:time.Time{sec:63571016044, nsec:658020236, loc:(*time.Location)(0x1ba6120)}}, Count:1}': 'the server has asked for the client to provide credentials (post events)' (will not retry!)

3 个答案:

答案 0 :(得分：22)

kubelet日志文件的前两行实际上指向了潜在的问题 - 您没有为kubelet指定任何客户端凭据以连接到主服务器。

kubelet的let u16data: [UInt16] = [0xD83D,0xDC36] //or let u16data = "Hello, ".utf16 var g = u16data.generate() var s: String = "" var utf16 = UTF16() while case let .Result(scalar) = utf16.decode(&g) { print(scalar, &s) } print(s) // prints 和--tls-cert-file参数用于配置kubelet上的http服务器（如果未指定，则为kubelet will generate a self-signed certificate for its https endpoint）。此证书/密钥对不用作提供给主服务器进行身份验证的客户端证书。

要指定凭据，有两个选项：kubeconfig文件和kubernetes_auth文件。后者已弃用，因此我建议使用kubeconfig文件。

在kubeconfig文件中，您需要指定kubelet应该向apiserver提供的承载令牌或客户端证书。您还可以为apiserver指定CA证书（如果您希望连接安全）或告诉kubelet跳过检查apiserver提供的证书。由于您拥有apiserver的证书，我建议您将CA证书添加到kubeconfig文件中。

kubeconfig文件应如下所示：

--tls-private-key-file

要生成base64编码的客户端证书，您应该能够运行类似apiVersion: v1 kind: Config users: - name: kubelet user: client-certificate-data: <base64-encoded-cert> client-key-data: <base64-encoded-key> clusters: - name: local cluster: certificate-authority-data: <base64-encoded-ca-cert> contexts: - context: cluster: local user: kubelet name: service-account-context current-context: service-account-context 的内容。如果您没有方便的CA证书，可以将cat /var/run/kubernetes/kubelet_36kr.pem | base64替换为certificate-authority-data: <base64-encoded-ca-cert>。

如果您将此文件放在insecure-skip-tls-verify: true，则应自动获取该文件。否则，您可以使用/var/lib/kubelet/kubeconfig参数指定自定义位置。

答案 1 :(得分：2)

所有归功于jnoller ，因为他在此issue上指定了以下命令。
他只是犯了一个错误，因为他运行kubectl config set-credentials两次

这类似于Robert Bailey已接受的答案，除非您不需要任何基础64，这样可以轻松编写脚本。

kubectl config set-cluster default-cluster --server=https://${MASTER} \
    --certificate-authority=/path/to/ca.pem 

kubectl config set-credentials default-admin \
    --certificate-authority=/path/to/ca.pem \
    --client-key=/path/to/admin-key.pem \
    --client-certificate=/path/to/admin.pem      

kubectl config set-context default-system --cluster=default-cluster --user=default-admin
kubectl config use-context default-system

在〜/ .kube / config 中生成的结果配置如下所示：

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/certs/ca.crt
    server: https://kubernetesmaster
  name: default-cluster
contexts:
- context:
    cluster: default-cluster
    user: default-admin
  name: default-system
current-context: default-system
kind: Config
preferences: {}
users:
- name: default-admin
  user:
    client-certificate: /etc/kubernetes/certs/server.crt
    client-key: /etc/kubernetes/certs/server.key

答案 2 :(得分：1)

正如Robert Bailey所述，主要问题与客户端凭据有关，以连接主服务器（＆＃34;无法加载kubeconfig文件/var/lib/kubelet/kubeconfig..." ）。 我没有手动创建kubeconfig文件，而是选择使用kubectl工具生成它。

来自docs的示例：

$ kubectl config set-credentials myuser --username=myusername --password=mypassword
$ kubectl config set-cluster local-server --server=http://localhost:8080
$ kubectl config set-context default-context --cluster=local-server --user=myuser
$ kubectl config use-context default-context
$ kubectl config set contexts.default-context.namespace mynamespace

这些命令将在〜/ .kube / config

中生成配置文件

检查结果：

$ kubectl config view

然后我在/ var / lib / kubelet（默认位置）里面创建了一个符号链接到我的配置文件：

$ cd /var/lib/kubelet
$ sudo ln -s ~/.kube/config kubeconfig

这对我有用。我希望它也适合你。

相关问题
最新问题