如何在服务节点端口范围默认范围之外的NodePort上启动kubernetes服务?

时间:2017-02-24 13:41:00

标签: azure docker kubernetes centos7

我一直在尝试在默认端口范围之外的NodePort上启动kubernetes-dashboard(以及eventualy其他服务),但收效甚微, 这是我的设置: 云提供商:Azure(非天蓝色容器服务) 操作系统:CentOS 7

这是我尝试过的:

更新主机

$ yum update

安装kubeadm

$ cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
       https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
$ setenforce 0
$ yum install -y docker kubelet kubeadm kubectl kubernetes-cni
$ systemctl enable docker && systemctl start docker
$ systemctl enable kubelet && systemctl start kubelet

使用kubeadm

启动集群
$ kubeadm init

允许在主节点上运行容器,因为我们有一个节点集群

$ kubectl taint nodes --all dedicated-

安装pod网络

$ kubectl apply -f https://git.io/weave-kube

我们的kubernetes-dashboard部署(@〜/ kubernetes-dashboard.yaml

# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or     implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Configuration to deploy release version of the Dashboard UI.
#
# Example usage: kubectl create -f <this_file>

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kubernetes-dashboard
  template:
    metadata:
      labels:
        app: kubernetes-dashboard
      # Comment the following annotation if Dashboard must not be deployed on master
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
      containers:
      - name: kubernetes-dashboard
        image: gcr.io/google_containers/kubernetes-dashboard-amd64:v1.5.1
        imagePullPolicy: Always
        ports:
        - containerPort: 9090
          protocol: TCP
        args:
          # Uncomment the following line to manually specify Kubernetes API server Host
          # If not specified, Dashboard will attempt to auto discover the API server and connect
          # to it. Uncomment only if the default does not work.
          # - --apiserver-host=http://my-address:port
        livenessProbe:
          httpGet:
            path: /
            port: 9090
          initialDelaySeconds: 30
          timeoutSeconds: 30
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - port: 8880
    targetPort: 9090
    nodePort: 8880
  selector:
    app: kubernetes-dashboard

创建我们的部署

$ kubectl create -f ~/kubernetes-dashboard.yaml
deployment "kubernetes-dashboard" created
The Service "kubernetes-dashboard" is invalid: spec.ports[0].nodePort: Invalid value: 8880: provided port is not in the valid range. The range of valid ports is 30000-32767

我发现要更改有效端口的范围,我可以在kube-apiserver上设置service-node-port-range选项以允许不同的端口范围, 所以我试过了:

$ kubectl get po --namespace=kube-system
NAME                                    READY     STATUS    RESTARTS       AGE
dummy-2088944543-lr2zb                  1/1       Running   0              31m
etcd-test2-highr                        1/1       Running   0              31m
kube-apiserver-test2-highr              1/1       Running   0              31m
kube-controller-manager-test2-highr     1/1       Running   2              31m
kube-discovery-1769846148-wmbhb         1/1       Running   0              31m
kube-dns-2924299975-8vwjm               4/4       Running   0              31m
kube-proxy-0ls9c                        1/1       Running   0              31m
kube-scheduler-test2-highr              1/1       Running   2              31m
kubernetes-dashboard-3203831700-qrvdn   1/1       Running   0              22s
weave-net-m9rxh                         2/2       Running   0              31m

将“--service-node-port-range = 8880-8880”添加到kube-apiserver-test2-highr

$ kubectl edit po kube-apiserver-test2-highr --namespace=kube-system
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-apiserver",
    "namespace": "kube-system",
    "creationTimestamp": null,
    "labels": {
      "component": "kube-apiserver",
      "tier": "control-plane"
    }
  },
  "spec": {
    "volumes": [
      {
        "name": "k8s",
        "hostPath": {
          "path": "/etc/kubernetes"
        }
      },
      {
        "name": "certs",
        "hostPath": {
          "path": "/etc/ssl/certs"
        }
      },
      {
        "name": "pki",
        "hostPath": {
          "path": "/etc/pki"
        }
      }
    ],
    "containers": [
      {
        "name": "kube-apiserver",
        "image": "gcr.io/google_containers/kube-apiserver-amd64:v1.5.3",
        "command": [
          "kube-apiserver",
          "--insecure-bind-address=127.0.0.1",
          "--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota",
          "--service-cluster-ip-range=10.96.0.0/12",
          "--service-node-port-range=8880-8880",
          "--service-account-key-file=/etc/kubernetes/pki/apiserver-key.pem",
          "--client-ca-file=/etc/kubernetes/pki/ca.pem",
          "--tls-cert-file=/etc/kubernetes/pki/apiserver.pem",
          "--tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem",
          "--token-auth-file=/etc/kubernetes/pki/tokens.csv",
          "--secure-port=6443",
          "--allow-privileged",
          "--advertise-address=100.112.226.5",
          "--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname",
          "--anonymous-auth=false",
          "--etcd-servers=http://127.0.0.1:2379"
        ],
        "resources": {
          "requests": {
            "cpu": "250m"
          }
        },
        "volumeMounts": [
          {
            "name": "k8s",
            "readOnly": true,
            "mountPath": "/etc/kubernetes/"
          },
          {
            "name": "certs",
            "mountPath": "/etc/ssl/certs"
          },
          {
            "name": "pki",
            "mountPath": "/etc/pki"
          }
        ],
        "livenessProbe": {
          "httpGet": {
            "path": "/healthz",
            "port": 8080,
            "host": "127.0.0.1"
          },
          "initialDelaySeconds": 15,
          "timeoutSeconds": 15,
          "failureThreshold": 8
        }
      }
    ],
    "hostNetwork": true
  },
  "status": {}

$ :wq

以下是截断的回复

# pods "kube-apiserver-test2-highr" was not valid:
# * spec: Forbidden: pod updates may not change fields other than `containers[*].image` or `spec.activeDeadlineSeconds`

所以我尝试了另一种方法,我使用上述相同的更改编辑了kube-apiserver的部署文件 并运行以下内容:

$ kubectl apply -f /etc/kubernetes/manifests/kube-apiserver.json --namespace=kube-system

得到了这个回复:

The connection to the server localhost:8080 was refused - did you specify the right host or port?

所以现在我被卡住了,我怎样才能改变有效端口的范围?

3 个答案:

答案 0 :(得分:3)

您指定--service-node-port-range=8880-8880错误。您只将其设置为一个端口,将其设置为范围。

第二个问题:您将服务设置为使用9090且不在范围内。

 ports:
  - port: 80
    targetPort: 9090
    nodePort: 9090

API服务器也应该有部署,尝试编辑部署本身的端口范围并删除api服务器pod,以便通过新配置重新创建它。

答案 1 :(得分:0)

出于某种原因,服务节点端口范围设置为不常用的端口。为什么要在每个节点上发布它?你真的想要吗?

另一种方法是在半随机节点端口上公开它,然后使用已知节点或节点集上的代理窗格通过hostport访问它。

答案 2 :(得分:0)

这个问题:

The connection to the server localhost:8080 was refused - did you specify the right host or port?

是由我的端口范围引起的,不包括8080,kube-apiserver正在服务,因此我无法向kubectl发送任何更新。

我通过将端口范围更改为8080-8881并重新启动kubelet服务来修复它:

$ service kubelet restart

现在一切正常。