Question

我们的Cluster中有两个Kafka Broker和3个zookeeper。

随机（2-10小时）经纪人正在走下坡路。

任何帮助都可以通过调试来解决问题。

日志

[2014-10-28 08:19:56,465] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
[2014-10-28 08:19:56,470] INFO [Socket Server on Broker 1], Shutting down (kafka.network.SocketServer)
[2014-10-28 08:19:56,477] INFO [Socket Server on Broker 1], Shutdown completed (kafka.network.SocketServer)
[2014-10-28 08:19:56,479] INFO [Kafka Request Handler on Broker 1], shutting down (kafka.server.KafkaRequestHandlerPool)
[2014-10-28 08:19:56,482] INFO [Kafka Request Handler on Broker 1], shut down completely (kafka.server.KafkaRequestHandlerPool)
[2014-10-28 08:19:56,600] INFO [Replica Manager on Broker 1]: Shut down (kafka.server.ReplicaManager)
[2014-10-28 08:19:56,601] INFO [ReplicaFetcherManager on broker 1] shutting down (kafka.server.ReplicaFetcherManager)
[2014-10-28 08:19:56,603] INFO [ReplicaFetcherThread-0-2], Shutting down (kafka.server.ReplicaFetcherThread)
[2014-10-28 08:19:56,993] INFO [ReplicaFetcherThread-0-2], Stopped  (kafka.server.ReplicaFetcherThread)
[2014-10-28 08:19:56,993] INFO [ReplicaFetcherThread-0-2], Shutdown completed (kafka.server.ReplicaFetcherThread)
[2014-10-28 08:19:56,997] INFO [ReplicaFetcherManager on broker 1] shutdown completed (kafka.server.ReplicaFetcherManager)
[2014-10-28 08:19:56,999] INFO [Replica Manager on Broker 1]: Shutted down completely (kafka.server.ReplicaManager)
[2014-10-28 08:19:57,031] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer)

GC日志

2014-10-28T23:20:58.503-0400: 2.044: [GC [1 CMS-initial-mark: 0K(699072K)] 156626K(1013632K), 0.1555280 secs] [Times: user=0.18 sys=0.00, real=0.15 secs]
2014-10-28T23:20:58.659-0400: 2.200: [CMS-concurrent-mark-start]
2014-10-28T23:20:58.684-0400: 2.225: [CMS-concurrent-mark: 0.011/0.025 secs] [Times: user=0.13 sys=0.01, real=0.03 secs]
2014-10-28T23:20:58.684-0400: 2.225: [CMS-concurrent-preclean-start]
2014-10-28T23:20:58.720-0400: 2.262: [CMS-concurrent-preclean: 0.035/0.037 secs] [Times: user=0.15 sys=0.00, real=0.03 secs]
2014-10-28T23:20:58.720-0400: 2.262: [CMS-concurrent-abortable-preclean-start]
 CMS: abort preclean due to time 2014-10-28T23:21:03.751-0400: 7.292: [CMS-concurrent-abortable-preclean: 0.165/5.030 secs] [Times: user=1.31 sys=0.04, real=5.03 secs]
2014-10-28T23:21:03.751-0400: 7.292: [GC[YG occupancy: 234944 K (314560 K)]2014-10-28T23:21:03.751-0400: 7.293: [GC2014-10-28T23:21:03.751-0400: 7.293: [ParNew: 234944K->10098K(314560K), 0.0168070 secs] 234944K->10098K(1013632K), 0.0169590 secs] [Times: user=0.10 sys=0.02, real=0.02 secs]
2014-10-28T23:21:03.768-0400: 7.310: [Rescan (parallel) , 0.0174650 secs]2014-10-28T23:21:03.786-0400: 7.327: [weak refs processing, 0.0000410 secs]2014-10-28T23:21:03.786-0400: 7.327: [class unloading, 0.0026070 secs]2014-10-28T23:21:03.789-0400: 7.330: [scrub symbol table, 0.0025960 secs]2014-10-28T23:21:03.791-0400: 7.332: [scrub string table, 0.0003650 secs] [1 CMS-remark: 0K(699072K)] 10098K(1013632K), 0.0407100 secs] [Times: user=0.28 sys=0.02, real=0.04 secs]
2014-10-28T23:21:03.795-0400: 7.336: [CMS-concurrent-sweep-start]
2014-10-28T23:21:03.804-0400: 7.345: [CMS-concurrent-sweep: 0.009/0.009 secs] [Times: user=0.03 sys=0.00, real=0.01 secs]
2014-10-28T23:21:03.804-0400: 7.346: [CMS-concurrent-reset-start]
2014-10-28T23:21:03.810-0400: 7.351: [CMS-concurrent-reset: 0.005/0.005 secs] [Times: user=0.02 sys=0.00, real=0.00 secs]
~
~

Answer 1

你的关机看起来很干净。检查已分配给JVM的内存量，并检查gc_dump。

我们在我们的环境中观察到，由于内存不足，JVM不断死亡，直到我们提高内存以满足JVM在我们环境中的需求。

卡夫卡经纪人 - 清洁关闭

1 个答案: