Question

我在我的kubernetes集群中使用private Docker registry addon，我想在每个节点上公开端口5000，以便轻松地从localhost：5000中提取图像。所以我在每个节点上放置一个pod清单文件/etc/kubernetes/manifests/kube-registry-proxy.manifest来启动端口5000的本地代理。几个月前我在裸机ubuntu上手动部署kubernetes时工作，但是当我尝试kargo失败时，端口5000没有听着。

我正在使用带有calico网络插件的kargo，docker注册表的配置是：

kind: PersistentVolume
apiVersion: v1
metadata:
  name: kube-system-kube-registry-pv
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  capacity:
    storage: 500Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: /registry

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: kube-registry-pvc
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 500Gi

apiVersion: v1
kind: ReplicationController
metadata:
  name: kube-registry-v0
  namespace: kube-system
  labels:
    k8s-app: kube-registry
    version: v0
    kubernetes.io/cluster-service: "true"
spec:
  replicas: 1
  selector:
    k8s-app: kube-registry
    version: v0
  template:
    metadata:
      labels:
        k8s-app: kube-registry
        version: v0
        kubernetes.io/cluster-service: "true"
    spec:
      containers:
      - name: registry
        image: registry:2.5.1
        resources:
          # keep request = limit to keep this container in guaranteed class
          limits:
            cpu: 100m
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: REGISTRY_HTTP_ADDR
          value: :5000
        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
          value: /var/lib/registry
        volumeMounts:
        - name: image-store
          mountPath: /var/lib/registry
        ports:
        - containerPort: 5000
          name: registry
          protocol: TCP
      volumes:
      - name: image-store
        persistentVolumeClaim:
          claimName: kube-registry-pvc

apiVersion: v1
kind: Service
metadata:
  name: kube-registry
  namespace: kube-system
  labels:
    k8s-app: kube-registry
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "KubeRegistry"
spec:
  selector:
    k8s-app: kube-registry
  ports:
  - name: registry
    port: 5000
    protocol: TCP

我在运行kargo之前创建了一个pod清单文件/etc/kubernetes/manifests/kube-registry-proxy.manifest：

apiVersion: v1
kind: Pod
metadata:
  name: kube-registry-proxy
  namespace: kube-system
spec:
  containers:
  - name: kube-registry-proxy
    image: gcr.io/google_containers/kube-registry-proxy:0.3
    resources:
      limits:
        cpu: 100m
        memory: 50Mi
    env:
    - name: REGISTRY_HOST
      value: kube-registry.kube-system.svc.cluster.local
    - name: REGISTRY_PORT
      value: "5000"
    - name: FORWARD_PORT
      value: "5000"
    ports:
    - name: registry
      containerPort: 5000
      hostPort: 5000

kube-registry-proxy正在所有节点上运行，但没有任何东西在端口5000上监听。有些输出：

ubuntu@k8s15m1:~$ kubectl get all --all-namespaces | grep registry-proxy
kube-system   po/kube-registry-proxy-k8s15m1             1/1       Running             1          1h
kube-system   po/kube-registry-proxy-k8s15m2             1/1       Running             0          1h
kube-system   po/kube-registry-proxy-k8s15s1             1/1       Running             0          1h

ubuntu@k8s15m1:~$ docker ps | grep registry
756fcf674288        gcr.io/google_containers/kube-registry-proxy:0.3      "/usr/bin/run_proxy"     19 minutes ago      Up 19 minutes                           k8s_kube-registry-proxy.bebf6da1_kube-registry-proxy-k8s15m1_kube-system_a818b22dc7210ecd31414e328ae28e43_7221833c

ubuntu@k8s15m1:~$ docker logs 756fcf674288 | tail
waiting for kube-registry.kube-system.svc.cluster.local to come online
starting proxy

ubuntu@k8s15m1:~$ netstat -ltnp | grep 5000

ubuntu@k8s15m1:~$ curl -v localhost:5000/v1/
*   Trying 127.0.0.1...
* connect to 127.0.0.1 port 5000 failed: Connection refused
* Failed to connect to localhost port 5000: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 5000: Connection refused

ubuntu@k8s15m1:~$ kubectl get po kube-registry-proxy-k8s15m1 --namespace=kube-system -o wide
NAME                          READY     STATUS    RESTARTS   AGE       IP             NODE
kube-registry-proxy-k8s15m1   1/1       Running   3          1h        10.233.69.64   k8s15m1

ubuntu@k8s15m1:~$ curl -v 10.233.69.64:5000/v1/
*   Trying 10.233.69.64...
* Connected to 10.233.69.64 (10.233.69.64) port 5000 (#0)
> GET /v1/ HTTP/1.1
> Host: 10.233.69.64:5000
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< Docker-Distribution-Api-Version: registry/2.0
< X-Content-Type-Options: nosniff
< Date: Tue, 14 Mar 2017 16:41:56 GMT
< Content-Length: 19
<
404 page not found
* Connection #0 to host 10.233.69.64 left intact

Answer 1

我认为这里有几件事情。

最重要的是，请注意Kubernetes Services有3种口味：ClusterIP（默认值），NodePort（听起来非常像您期望的那样），和LoadBalancer（我不会进一步提及，但文档会这样做。）

我希望如果您将Service更新为明确请求type: NodePort，您就会更接近您的想法（但请注意，除非您更改它，{{3} }。

因此：

apiVersion: v1 kind: Service metadata: name: kube-registry namespace: kube-system labels: k8s-app: kube-registry kubernetes.io/cluster-service: "true" kubernetes.io/name: "KubeRegistry" spec: type: NodePort # <--- updated line selector: k8s-app: kube-registry ports: - name: registry port: 5000 nodePort: 30500 # <-- you can specify, or omit this protocol: TCP

如果您对要服务的端口有意见，请随意指定，或者将其关闭，Kubernetes将从可用空间中选择一个。

我要提到接下来的事情是为了完整，但我要说的是一个不好的做法，所以请不要这样做。您还可以Pods直接在Node的实际TCP / IP堆栈上监听NodePort ports are limited to 30000-32767;所以在你的情况下，hostPort: 5000就在containerPort: 5000之下，导致Pod表现得像普通的docker -p 5000:5000命令。但这样做会使Pods计划成为一场噩梦，所以请不要这样做。

其次，关于来自curl的您的404：

我将假设，基于curl命令的输出，10.233.69.x是您的服务CIDR，这解释了为什么端口5000响应了什么。该请求是正确的，但/v1/是一个不正确的URI尝试。 specifying hostPort包含有关Docker Registry API docs的部分。我最喜欢的curl注册表是https://registry.example.com/v2/_catalog，因为它会返回其中每个图像的名称，确保我的凭据正确，注册服务器正常运行，等等。

我知道要接受很多东西，所以如果你觉得我掩饰了什么，请告诉我，我会尽力解决它。祝你好运！

kube-registry-proxy不会在任何节点上公开端口5000

1 个答案: