卡夫卡消费者在特定偏移量上断开连接

时间:2019-02-05 12:30:16

标签: java apache-kafka kafka-consumer-api

我正在运行kafka docker image。我目睹了消费者在相同的特定偏移量上断开连接。 尝试重新启动我的使用者并从该偏移量开始,但是仍然存在相同的问题:

Feb 05, 2019 1:28:45 PM org.apache.kafka.clients.FetchSessionHandler handleError
INFO: [Consumer clientId=consumer-1, groupId=integration_AAAA] Error sending fetch request (sessionId=1477368817, epoch=1) to node 1001: org.apache.kafka.common.errors.DisconnectException.
Feb 05, 2019 1:28:45 PM org.apache.kafka.clients.FetchSessionHandler handleError
INFO: [Consumer clientId=consumer-1, groupId=integration_AAAA] Error sending fetch request (sessionId=1477368817, epoch=INITIAL) to node 1001: org.apache.kafka.common.errors.DisconnectException.
Feb 05, 2019 1:28:45 PM org.apache.kafka.clients.FetchSessionHandler handleError
INFO: [Consumer clientId=consumer-1, groupId=integration_AAAA] Error sending fetch request (sessionId=1477368817, epoch=INITIAL) to node 1001: org.apache.kafka.common.errors.DisconnectException.
Feb 05, 2019 1:28:45 PM org.apache.kafka.clients.FetchSessionHandler handleError

以下是交易/调试日志:

2019-02-05 13:54:10,067 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Found least loaded node kafka:9092 (id: 1001 rack: null)
2019-02-05 13:54:10,071 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Sending FIND_COORDINATOR {coordinator_key=integration_AAAA,coordinator_type=0} with correlation id 5 to node 1001
2019-02-05 13:54:24,823 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Disconnecting from node 1001 due to request timeout.
2019-02-05 13:54:24,825 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Cancelled request FIND_COORDINATOR {coordinator_key=integration_AAAA,coordinator_type=0} with correlation id 5 due to node 1001 being disconnected
2019-02-05 13:54:24,825 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Cancelled request FETCH {replica_id=-1,max_wait_time=500,min_bytes=1,max_bytes=52428800,isolation_level=0,session_id=149477758,epoch=1,topics=[{topic=myTopic,partitions=[{partition=0,fetch_offset=19666,log_start_offset=-1,max_bytes=1048576}]}],forgetten_topics_data=[]} with correlation id 4 due to node 1001 being disconnected
2019-02-05 13:54:24,827 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,827 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,828 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,829 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,829 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,829 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,829 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,829 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,830 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,830 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,830 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,830 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,831 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,831 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,831 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,831 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,832 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,832 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,832 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,832 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,833 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,833 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,833 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,833 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,833 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,834 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,834 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,834 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,834 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,834 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available
2019-02-05 13:54:24,835 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Removing node kafka:9092 (id: 1001 rack: null) from least loaded node selection: is-blacked-out: true, in-flight-requests: 0
2019-02-05 13:54:24,835 [main] TRACE org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Least loaded node selection failed to find an available node
2019-02-05 13:54:24,835 [main] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=integration_AAAA] Give up sending metadata request since no node is available

请问关于相同偏移量导致此问题的原因是什么? 谢谢,

0 个答案:

没有答案