发送消息时,部署在kuberenetees中的Kafka重新启动

时间:2020-10-24 14:10:09

标签: networking kubernetes deployment apache-kafka kafka-consumer-api

我使用Debezium泊坞映像在kubernetes中部署了Kafka,Zookeeper和Kafka Connect。它们看起来不错,但是当我尝试使用控制台使用者和生产者来测试发送和接收消息时,kafka部署会重新启动。 首先,我开始消费者:

bin/kafka-console-consumer.sh --topic topic --from-beginning --bootstrap-server cluster-IP:9092

然后是制作人:

bin/kafka-console-producer.sh --topic topic --bootstrap-server cluster-IP:9092

到目前为止,一切似乎都还不错。使用者正在等待消息,而生产者正在等待键入消息。但是,当我尝试输入任何消息并将其发送到生产者中时,kafka部署会重新启动。

我可能只是添加,当我尝试列出主题时,一切正常,并且可以使用以下命令查看主题名称:

bin/kafka-topics.sh --list --zookeeper cluster-IP:2181

在重新启动后,当我从主题开始尝试使用使用者时,也没有消息。

这是最近的日志:


2020-10-24 13:49:16,421 - DEBUG [Controller-1-to-broker-1-send-thread:NetworkClient@840] - [Controller id=1, targetBrokerId=1] Received UPDATE_METADATA response from node 1 for request with header RequestHeader(apiKey=UPDATE_METADATA, apiVersion=6, clientId=1, correlationId=2): org.apache.kafka.common.requests.UpdateMetadataResponse@78b355b9
2020-10-24 13:49:21,437 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 0ms
2020-10-24 13:49:27,441 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 0ms
2020-10-24 13:49:33,445 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 0ms
2020-10-24 13:49:39,451 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 0ms
2020-10-24 13:49:45,459 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 1ms
2020-10-24 13:49:51,465 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 1ms
2020-10-24 13:49:57,465 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 0ms
2020-10-24 13:50:03,471 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 0ms
2020-10-24 13:50:09,522 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 1ms
2020-10-24 13:50:15,528 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 2ms
2020-10-24 13:50:21,624 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 4ms
2020-10-24 13:50:22,720 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@199] - [SocketServer brokerId=1] clients: Adding a new metric ClientInformation(softwareName=unknown, softwareVersion=unknown).
2020-10-24 13:50:22,721 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@213] - [SocketServer brokerId=1] clients: entering performPendingMetricsOperations
2020-10-24 13:50:22,721 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@224] - [SocketServer brokerId=1] clients: adding metric MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=unknown, clientSoftwareVersion=unknown, listener=PLAINTEXT, networkProcessor=1}]
2020-10-24 13:50:22,721 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:Metrics@587] - Registered metric named MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=unknown, clientSoftwareVersion=unknown, listener=PLAINTEXT, networkProcessor=1}]
2020-10-24 13:50:22,721 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@229] - [SocketServer brokerId=1] clients: leaving performPendingMetricsOperations
2020-10-24 13:50:23,024 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@247] - [SocketServer brokerId=1] clients: Removed a reference to ClientInformation(softwareName=unknown, softwareVersion=unknown).  0 reference(s) remaining.
2020-10-24 13:50:23,024 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@199] - [SocketServer brokerId=1] clients: Adding a new metric ClientInformation(softwareName=apache-kafka-java, softwareVersion=2.6.0).
2020-10-24 13:50:23,024 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@213] - [SocketServer brokerId=1] clients: entering performPendingMetricsOperations
2020-10-24 13:50:23,024 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@224] - [SocketServer brokerId=1] clients: adding metric MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=apache-kafka-java, clientSoftwareVersion=2.6.0, listener=PLAINTEXT, networkProcessor=1}]
2020-10-24 13:50:23,025 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:Metrics@587] - Registered metric named MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=apache-kafka-java, clientSoftwareVersion=2.6.0, listener=PLAINTEXT, networkProcessor=1}]
2020-10-24 13:50:23,025 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-1:IntGaugeSuite@229] - [SocketServer brokerId=1] clients: leaving performPendingMetricsOperations
2020-10-24 13:50:24,422 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@199] - [SocketServer brokerId=1] clients: Adding a new metric ClientInformation(softwareName=unknown, softwareVersion=unknown).
2020-10-24 13:50:24,422 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@213] - [SocketServer brokerId=1] clients: entering performPendingMetricsOperations
2020-10-24 13:50:24,422 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@224] - [SocketServer brokerId=1] clients: adding metric MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=unknown, clientSoftwareVersion=unknown, listener=PLAINTEXT, networkProcessor=2}]
2020-10-24 13:50:24,422 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:Metrics@587] - Registered metric named MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=unknown, clientSoftwareVersion=unknown, listener=PLAINTEXT, networkProcessor=2}]
2020-10-24 13:50:24,422 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@229] - [SocketServer brokerId=1] clients: leaving performPendingMetricsOperations
2020-10-24 13:50:24,630 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@247] - [SocketServer brokerId=1] clients: Removed a reference to ClientInformation(softwareName=unknown, softwareVersion=unknown).  0 reference(s) remaining.
2020-10-24 13:50:24,630 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@199] - [SocketServer brokerId=1] clients: Adding a new metric ClientInformation(softwareName=apache-kafka-java, softwareVersion=2.6.0).
2020-10-24 13:50:24,630 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@213] - [SocketServer brokerId=1] clients: entering performPendingMetricsOperations
2020-10-24 13:50:24,630 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@224] - [SocketServer brokerId=1] clients: adding metric MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=apache-kafka-java, clientSoftwareVersion=2.6.0, listener=PLAINTEXT, networkProcessor=2}]
2020-10-24 13:50:24,630 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:Metrics@587] - Registered metric named MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=apache-kafka-java, clientSoftwareVersion=2.6.0, listener=PLAINTEXT, networkProcessor=2}]
2020-10-24 13:50:24,630 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-2:IntGaugeSuite@229] - [SocketServer brokerId=1] clients: leaving performPendingMetricsOperations
2020-10-24 13:50:25,231 - INFO  [data-plane-kafka-request-handler-2:Logging@66] - [GroupCoordinator 1]: Preparing to rebalance group console-consumer-88244 in state PreparingRebalance with old generation 0 (__consumer_offsets-37) (reason: Adding new member consumer-console-consumer-88244-1-4d8b1104-ffeb-454a-8730-0ef3aa554c1e with group instance id None)
2020-10-24 13:50:25,728 - INFO  [executor-Rebalance:Logging@66] - [GroupCoordinator 1]: Stabilized group console-consumer-88244 generation 1 (__consumer_offsets-37)
2020-10-24 13:50:25,932 - INFO  [data-plane-kafka-request-handler-3:Logging@66] - [GroupCoordinator 1]: Assignment received from leader for group console-consumer-88244 for generation 1
2020-10-24 13:50:27,722 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 98ms
2020-10-24 13:50:28,221 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:IntGaugeSuite@247] - [SocketServer brokerId=1] clients: Removed a reference to ClientInformation(softwareName=unknown, softwareVersion=unknown).  1 reference(s) remaining.
2020-10-24 13:50:28,221 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:IntGaugeSuite@199] - [SocketServer brokerId=1] clients: Adding a new metric ClientInformation(softwareName=apache-kafka-java, softwareVersion=2.6.0).
2020-10-24 13:50:28,221 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:IntGaugeSuite@213] - [SocketServer brokerId=1] clients: entering performPendingMetricsOperations
2020-10-24 13:50:28,221 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:IntGaugeSuite@224] - [SocketServer brokerId=1] clients: adding metric MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=apache-kafka-java, clientSoftwareVersion=2.6.0, listener=PLAINTEXT, networkProcessor=0}]
2020-10-24 13:50:28,221 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:Metrics@587] - Registered metric named MetricName [name=connections, group=socket-server-metrics, description=The number of connections with this client and version., tags={clientSoftwareName=apache-kafka-java, clientSoftwareVersion=2.6.0, listener=PLAINTEXT, networkProcessor=0}]
2020-10-24 13:50:28,221 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:IntGaugeSuite@229] - [SocketServer brokerId=1] clients: leaving performPendingMetricsOperations
2020-10-24 13:50:29,325 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 73, total bytes written so far: 73, expected bytes to write: 73
2020-10-24 13:50:29,933 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:30,523 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:31,028 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:31,625 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:32,134 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:32,725 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:33,321 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:33,720 - DEBUG [main-SendThread(zookeeper:2181):ClientCnxn$SendThread@821] - Got ping response for sessionid: 0x1001526a2a1002d after 90ms
2020-10-24 13:50:33,830 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:34,424 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:35,021 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:35,528 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22
2020-10-24 13:50:36,121 - TRACE [data-plane-kafka-network-thread-1-ListenerName(PLAINTEXT)-PLAINTEXT-0:MultiRecordsSend@107] - Bytes written as part of multi-send call: 22, total bytes written so far: 22, expected bytes to write: 22

0 个答案:

没有答案