Kubernetes上的Kafka-无法在Kubernetes之外生产/消费

时间:2019-01-14 11:45:16

标签: kubernetes apache-kafka

我能够从kubernetes内部产生/使用消息而没有任何错误 但是,我无法从kubernetes外部访问

这是我的设置:

kubectl get svc -o wide

kafka-test                         NodePort    10.108.201.154   <none>        9092:30989/TCP                  1d    app=kafka,release=kafka-test
kafka-test-0-external              NodePort    10.97.62.11      <none>        9092:31090/TCP                  1d    app=kafka,release=kafka-test,statefulset.kubernetes.io/pod-name=kafka-test-0
kafka-test-1-external              NodePort    10.99.211.234    <none>        9092:31091/TCP                  1d    app=kafka,release=kafka-test,statefulset.kubernetes.io/pod-name=kafka-test-1
kafka-test-2-external              NodePort    10.105.160.163   <none>        9092:31092/TCP                  1d    app=kafka,release=kafka-test,statefulset.kubernetes.io/pod-name=kafka-test-2
kafka-test-headless                ClusterIP   None             <none>        9092/TCP                        1d    app=kafka,release=kafka-test
kafka-test-zookeeper               NodePort    10.106.113.172   <none>        2181:31541/TCP                  1d    app=zookeeper,release=kafka-test

kubectl get pods -o wide

kafka-test-0                                        1/1     Running            0          1d    192.168.2.229    ip-xx-xx-xx-xx.ap-south-1.compute.internal     <none>
kafka-test-1                                        1/1     Running            0          1d    192.168.59.198   ip-xx-xx-xx-xx.ap-south-1.compute.internal     <none>
kafka-test-2                                        1/1     Running            0          1d    192.168.4.9      ip-xx-xx-xx-xx.ap-south-1.compute.internal   <none>
kafka-test-zookeeper-0                              1/1     Running            0          1d    192.168.2.228    ip-xx-xx-xx-xx.ap-south-1.compute.internal     <none>
kafka-test-zookeeper-1                              1/1     Running            0          1d    192.168.59.199   ip-xx-xx-xx-xx.ap-south-1.compute.internal     <none>
kafka-test-zookeeper-2                              1/1     Running            0          1d    192.168.4.10     ip-xx-xx-xx-xx.ap-south-1.compute.internal   <none>

我正在运行具有以下属性的3 kafka服务

/etc/kafka/bin/kafka-server-start.sh /etc/kafka/config/server.properties \
	  --override listener.security.protocol.map=INTERNAL_PLAINTEXT:PLAINTEXT,EXTERNAL_PLAINTEXT:PLAINTEXT \
      --override advertised.listeners=INTERNAL_PLAINTEXT://kafka-test-0.kafka-test-headless.spr-ops.svc.cluster.local:9092,EXTERNAL_PLAINTEXT://<publicdns name of pod instance>:31090 \
      --override listeners=INTERNAL_PLAINTEXT://0.0.0.0:9092,EXTERNAL_PLAINTEXT://0.0.0.0:31090 \
      --override inter.broker.listener.name=INTERNAL_PLAINTEXT

/etc/kafka/bin/kafka-server-start.sh /etc/kafka/config/server.properties \
	  --override listener.security.protocol.map=INTERNAL_PLAINTEXT:PLAINTEXT,EXTERNAL_PLAINTEXT:PLAINTEXT \
      --override advertised.listeners=INTERNAL_PLAINTEXT://kafka-test-1.kafka-test-headless.spr-ops.svc.cluster.local:9092,EXTERNAL_PLAINTEXT://<publicdns name of pod instance>:31091 \
      --override listeners=INTERNAL_PLAINTEXT://0.0.0.0:9092,EXTERNAL_PLAINTEXT://0.0.0.0:31091 \
      --override inter.broker.listener.name=INTERNAL_PLAINTEXT

/etc/kafka/bin/kafka-server-start.sh /etc/kafka/config/server.properties \
	  --override listener.security.protocol.map=INTERNAL_PLAINTEXT:PLAINTEXT,EXTERNAL_PLAINTEXT:PLAINTEXT \
      --override advertised.listeners=INTERNAL_PLAINTEXT://kafka-test-2.kafka-test-headless.spr-ops.svc.cluster.local:9092,EXTERNAL_PLAINTEXT://<publicdns name of pod instance>:31092 \
      --override listeners=INTERNAL_PLAINTEXT://0.0.0.0:9092,EXTERNAL_PLAINTEXT://0.0.0.0:31092 \
      --override inter.broker.listener.name=INTERNAL_PLAINTEXT

我能够使用开放给外部访问的端口31090、31091、31092远程登录到实例。

我能够从外部kubernetes列出,描述主题,但是生产/消费无法从外部进行

当我尝试从外部k8s进行生产时,我遇到了麻烦

./kafka-console-producer.sh --broker-list <publicdns name of pod instance>:31090,<publicdns name of pod instance>:31091,<publicdns name of pod instance>:31092 --topic spr

hi
[2019-01-14 11:38:50,623] ERROR Error when sending message to topic spr with key: null, value: 2 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for spr-1: 1524 ms has passed since batch creation plus linger time
test
[2019-01-14 11:40:24,385] ERROR Error when sending message to topic spr with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for spr-0: 1518 ms has passed since batch creation plus linger time

我一直坚持不懈,并尝试了很长时间,这里最能提供帮助的人。

0 个答案:

没有答案