是什么导致团队像永恒一样保持平衡？

时间：2018-09-12 14:58:17

标签： apache-spark apache-kafka

已经6个小时了，我停止运行从Kafka主题消耗的Spark Streaming作业，但是我仍然收到此错误：

$ <path_to>/bin/kafka-consumer-groups.sh --describe --group myNaughtyGroup --bootstrap-server someHost:6667
Consumer group `myNaughtyGroup` is rebalancing.

此外，在运行我的Spark作业时，我无休止地等待这一行日志，这意味着Spark微批处理已开始一项作业，但它没有出现：

JobScheduler org.apache.spark.streaming.scheduler.JobScheduler: Starting job streaming job

要与Kafka集成，该项目取决于以下API： https://spark.apache.org/docs/2.2.0/streaming-kafka-0-10-integration.html

使用的版本：

  • Kafka服务器版本：0.10.0

  • Kafka spark集成的应用程序版本：“ org.apache.spark” %%“ spark-streaming-kafka-0-10”％2.1.1

  • Spark应用程序版本：2.2.0

  • Spark服务器版本：2.1.1

日志:)，其中最后一行通过增加版本来重复：

DEBUG 12-09-2018 11:39:52,564 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name heartbeat-latency
DEBUG 12-09-2018 11:39:52,564 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name heartbeat-latency
DEBUG 12-09-2018 11:39:52,565 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name join-latency
DEBUG 12-09-2018 11:39:52,565 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name join-latency
DEBUG 12-09-2018 11:39:52,565 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name sync-latency
DEBUG 12-09-2018 11:39:52,565 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name sync-latency
DEBUG 12-09-2018 11:39:52,567 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name commit-latency
DEBUG 12-09-2018 11:39:52,567 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name commit-latency
DEBUG 12-09-2018 11:39:52,573 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name bytes-fetched
DEBUG 12-09-2018 11:39:52,573 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name bytes-fetched
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name records-fetched
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name records-fetched
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name fetch-latency
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name fetch-latency
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name records-lag
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name records-lag
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name fetch-throttle-time
DEBUG 12-09-2018 11:39:52,574 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name fetch-throttle-time
INFO  12-09-2018 11:39:52,576 ForkJoinPool-1-worker-29 org.apache.kafka.common.utils.AppInfoParser: Kafka version : 0.10.0.1
INFO  12-09-2018 11:39:52,576 ForkJoinPool-1-worker-29 org.apache.kafka.common.utils.AppInfoParser: Kafka version : 0.10.0.1
INFO  12-09-2018 11:39:52,576 ForkJoinPool-1-worker-29 org.apache.kafka.common.utils.AppInfoParser: Kafka commitId : a7a17cdec9eaa6c5
INFO  12-09-2018 11:39:52,576 ForkJoinPool-1-worker-29 org.apache.kafka.common.utils.AppInfoParser: Kafka commitId : a7a17cdec9eaa6c5
DEBUG 12-09-2018 11:39:52,577 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.KafkaConsumer: Kafka consumer created
DEBUG 12-09-2018 11:39:52,577 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.KafkaConsumer: Kafka consumer created
DEBUG 12-09-2018 11:39:52,577 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.KafkaConsumer: Subscribed to topic(s): myNaughtyTopic
DEBUG 12-09-2018 11:39:52,577 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.KafkaConsumer: Subscribed to topic(s): myNaughtyTopic
DEBUG 12-09-2018 11:39:52,578 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Sending coordinator request for group myNaughtyGroup to broker kafka-broker1-host:6667 (id: -1 rack: null)
DEBUG 12-09-2018 11:39:52,578 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Sending coordinator request for group myNaughtyGroup to broker kafka-broker1-host:6667 (id: -1 rack: null)
DEBUG 12-09-2018 11:39:52,594 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node -1 at kafka-broker1-host:6667.
DEBUG 12-09-2018 11:39:52,594 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node -1 at kafka-broker1-host:6667.
DEBUG 12-09-2018 11:39:52,597 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initialize connection to node -2 for sending metadata request
DEBUG 12-09-2018 11:39:52,597 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initialize connection to node -2 for sending metadata request
DEBUG 12-09-2018 11:39:52,597 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node -2 at kafka-broker2-host:6667.
DEBUG 12-09-2018 11:39:52,597 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node -2 at kafka-broker2-host:6667.
DEBUG 12-09-2018 11:39:52,598 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--1.bytes-sent
DEBUG 12-09-2018 11:39:52,598 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--1.bytes-sent
DEBUG 12-09-2018 11:39:52,598 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--1.bytes-received
DEBUG 12-09-2018 11:39:52,598 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--1.bytes-received
DEBUG 12-09-2018 11:39:52,598 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--1.latency
DEBUG 12-09-2018 11:39:52,598 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--1.latency
DEBUG 12-09-2018 11:39:52,599 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node -1
DEBUG 12-09-2018 11:39:52,599 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node -1
DEBUG 12-09-2018 11:39:52,600 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node -1
DEBUG 12-09-2018 11:39:52,600 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node -1
DEBUG 12-09-2018 11:39:52,600 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--2.bytes-sent
DEBUG 12-09-2018 11:39:52,600 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--2.bytes-sent
DEBUG 12-09-2018 11:39:52,600 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--2.bytes-received
DEBUG 12-09-2018 11:39:52,600 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--2.bytes-received
DEBUG 12-09-2018 11:39:52,601 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--2.latency
DEBUG 12-09-2018 11:39:52,601 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node--2.latency
DEBUG 12-09-2018 11:39:52,601 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node -2
DEBUG 12-09-2018 11:39:52,601 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node -2
DEBUG 12-09-2018 11:39:52,615 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 2 to Cluster(nodes = [kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ), kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:39:52,615 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 2 to Cluster(nodes = [kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ), kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:39:52,616 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received group coordinator response ClientResponse(receivedTimeMs=1536752392615, disconnected=false, request=ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@699ce386, request=RequestSend(header={api_key=10,api_version=0,correlation_id=0,client_id=consumer-1}, body={group_id=myNaughtyGroup}), createdTimeMs=1536752392591, sendTimeMs=1536752392601), responseBody={error_code=0,coordinator={node_id=3005,host=kafka-broker1-host,port=6667}})
DEBUG 12-09-2018 11:39:52,616 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received group coordinator response ClientResponse(receivedTimeMs=1536752392615, disconnected=false, request=ClientRequest(expectResponse=true, callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@699ce386, request=RequestSend(header={api_key=10,api_version=0,correlation_id=0,client_id=consumer-1}, body={group_id=myNaughtyGroup}), createdTimeMs=1536752392591, sendTimeMs=1536752392601), responseBody={error_code=0,coordinator={node_id=3005,host=kafka-broker1-host,port=6667}})
INFO  12-09-2018 11:39:52,616 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Discovered coordinator kafka-broker1-host:6667 (id: 2147482643 rack: null) for group myNaughtyGroup.
INFO  12-09-2018 11:39:52,616 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Discovered coordinator kafka-broker1-host:6667 (id: 2147482643 rack: null) for group myNaughtyGroup.
DEBUG 12-09-2018 11:39:52,616 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node 2147482643 at kafka-broker1-host:6667.
DEBUG 12-09-2018 11:39:52,616 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node 2147482643 at kafka-broker1-host:6667.
INFO  12-09-2018 11:39:52,617 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.ConsumerCoordinator: Revoking previously assigned partitions [] for group myNaughtyGroup
INFO  12-09-2018 11:39:52,617 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.ConsumerCoordinator: Revoking previously assigned partitions [] for group myNaughtyGroup
INFO  12-09-2018 11:39:52,617 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: (Re-)joining group myNaughtyGroup
INFO  12-09-2018 11:39:52,617 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: (Re-)joining group myNaughtyGroup
DEBUG 12-09-2018 11:39:52,619 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Sending JoinGroup ({group_id=myNaughtyGroup,session_timeout=11400000,member_id=,protocol_type=consumer,group_protocols=[{protocol_name=range,protocol_metadata=java.nio.HeapByteBuffer[pos=0 lim=37 cap=37]}]}) to coordinator kafka-broker1-host:6667 (id: 2147482643 rack: null)
DEBUG 12-09-2018 11:39:52,619 ForkJoinPool-1-worker-29 org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Sending JoinGroup ({group_id=myNaughtyGroup,session_timeout=11400000,member_id=,protocol_type=consumer,group_protocols=[{protocol_name=range,protocol_metadata=java.nio.HeapByteBuffer[pos=0 lim=37 cap=37]}]}) to coordinator kafka-broker1-host:6667 (id: 2147482643 rack: null)
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-2147482643.bytes-sent
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-2147482643.bytes-sent
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-2147482643.bytes-received
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-2147482643.bytes-received
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-2147482643.latency
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-2147482643.latency
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node 2147482643
DEBUG 12-09-2018 11:39:52,620 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node 2147482643
INFO  12-09-2018 11:40:52,113 spark-dynamic-executor-allocation org.apache.spark.ExecutorAllocationManager: Request to remove executorIds: 1
DEBUG 12-09-2018 11:44:52,701 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initialize connection to node 3006 for sending metadata request
DEBUG 12-09-2018 11:44:52,701 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initialize connection to node 3006 for sending metadata request
DEBUG 12-09-2018 11:44:52,701 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node 3006 at kafka-broker2-host:6667.
DEBUG 12-09-2018 11:44:52,701 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node 3006 at kafka-broker2-host:6667.
DEBUG 12-09-2018 11:44:52,702 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initialize connection to node 3005 for sending metadata request
DEBUG 12-09-2018 11:44:52,702 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initialize connection to node 3005 for sending metadata request
DEBUG 12-09-2018 11:44:52,702 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node 3005 at kafka-broker1-host:6667.
DEBUG 12-09-2018 11:44:52,702 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Initiating connection to node 3005 at kafka-broker1-host:6667.
DEBUG 12-09-2018 11:44:52,702 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3006.bytes-sent
DEBUG 12-09-2018 11:44:52,702 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3006.bytes-sent
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3006.bytes-received
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3006.bytes-received
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3006.latency
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3006.latency
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node 3006
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node 3006
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3006
DEBUG 12-09-2018 11:44:52,703 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3006
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3005.bytes-sent
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3005.bytes-sent
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3005.bytes-received
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3005.bytes-received
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3005.latency
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.common.metrics.Metrics: Added sensor with name node-3005.latency
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node 3005
DEBUG 12-09-2018 11:44:52,704 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Completed connection to node 3005
DEBUG 12-09-2018 11:44:52,705 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 3 to Cluster(nodes = [kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ), kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:44:52,705 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 3 to Cluster(nodes = [kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ), kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:49:52,805 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3005
DEBUG 12-09-2018 11:49:52,805 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3005
DEBUG 12-09-2018 11:49:52,806 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 4 to Cluster(nodes = [kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ), kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:49:52,806 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 4 to Cluster(nodes = [kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ), kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:54:52,905 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3006
DEBUG 12-09-2018 11:54:52,905 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3006
DEBUG 12-09-2018 11:54:52,907 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 5 to Cluster(nodes = [kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ), kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:54:52,907 ForkJoinPool-1-worker-29 org.apache.kafka.clients.Metadata: Updated cluster metadata version 5 to Cluster(nodes = [kafka-broker1-host:6667 (id: 3005 rack: /CZ260802KZ), kafka-broker2-host:6667 (id: 3006 rack: /CZ260802KZ)], partitions = [Partition(topic = myNaughtyTopic, partition = 7, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 8, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 9, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 3, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 4, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 5, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 6, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 0, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 1, leader = 3005, replicas = [3005,], isr = [3005,], Partition(topic = myNaughtyTopic, partition = 2, leader = 3005, replicas = [3005,], isr = [3005,]])
DEBUG 12-09-2018 11:59:53,05 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3006
DEBUG 12-09-2018 11:59:53,05 ForkJoinPool-1-worker-29 org.apache.kafka.clients.NetworkClient: Sending metadata request {topics=[myNaughtyTopic]} to node 3006

如果您告诉我期望在日志中找到的内容，正常情况下Kafka的DEBUG级别以及上面的日志中的正常水平，那将很好。非常感谢！

0 个答案:

没有答案
相关问题
最新问题