Cassandra错误:检测到泄漏

时间:2017-06-19 00:17:38

标签: mysql cassandra cqlsh

我有一个双节点Cassandra集群(版本:3.10)。 我试图从SQL读取数据并写入Cassandra。一切都很好,但在某些时候我得到了这个错误 -

Exception in thread "main" com.datastax.driver.core.exceptions.TransportException: [/192.168.22.231:9042] Connection has been closed
        at com.datastax.driver.core.exceptions.TransportException.copy(TransportException.java:38)
        at com.datastax.driver.core.exceptions.TransportException.copy(TransportException.java:24)
        at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
        at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:43)
        at TableCreator.insertDataToCassandra(TableCreator.java:1037)
        at TableCreator.createTable(TableCreator.java:356)
        at DbMigration.main(DbMigration.java:25)
Caused by: com.datastax.driver.core.exceptions.TransportException: [/192.168.22.231:9042] Connection has been closed
        at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1215)
        at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1200)
        at com.datastax.driver.core.Connection.defunct(Connection.java:450)

Nodetool状态显示节点都是UP并且健康 -

Datacenter: dc1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address         Load       Tokens       Owns (effective)  Host ID                               Rack
UN  192.168.22.229  77.39 GiB  256          100.0%            335fc3a2-c21f-44ad-a937-487ba457c2fa  rack1
UN  192.168.22.231  77.39 GiB  256          100.0%            a5eaf96c-eaf9-4e2e-bd6b-6186ce944cd0  rack1

现在我无法连接到我的第一个节点 -

cqlsh --connect-timeout=30 192.168.22.231
Connection error: ('Unable to connect to any servers', {'192.168.22.231': error(111, "Tried connecting to [('192.168.22.231', 9042)]. Last error: Connection refused")})

但我可以连接到第二台服务器。我试着检查system.log和debug.log 的 syatem.log

  

INFO [CompactionExecutor:4003] 2017-06-19 07:25:19,676   NoSpamLogger.java:91 - 达到的最大内存使用量(1.000GiB),不能   分配1.000MiB INFO的块[IndexSummaryManager:1] 2017-06-19   07:28:02,222 IndexSummaryRedistribution.java:75 - 重新分发索引   摘要信息[CompactionExecutor:4009] 2017-06-19 07:40:42,023   NoSpamLogger.java:91 - 达到的最大内存使用量(1.000GiB),不能   分配1.000MiB信息块[CompactionExecutor:4015] 2017-06-19   07:56:04,582 NoSpamLogger.java:91 - 达到最大内存使用量   (1.000GiB),不能分配1.000MiB INFO的块   [CompactionExecutor:4021] 2017-06-19 08:11:26,674 NoSpamLogger.java:91    - 达到最大内存使用量(1.000GiB),不能分配1.000MiB信息块[服务主题] 2017-06-19 08:21:48,726 GCInspector.java:284 - ConcurrentMarkSweep GC在225ms。 CMS Old Gen:   5813642680 - > 3194404296; Par Eden Space:10360904 - > 347594616;平价   幸存者空间:83886080 - > 42514752

     

INFO [CompactionExecutor:4027] 2017-06-19 08:26:49,414   NoSpamLogger.java:91 - 达到的最大内存使用量(1.000GiB),不能   分配1.000MiB INFO的块[IndexSummaryManager:1] 2017-06-19   08:28:02,341 IndexSummaryRedistribution.java:75 - 重新分发索引   摘要信息[CompactionExecutor:4031] 2017-06-19 08:42:12,733   NoSpamLogger.java:91 - 达到的最大内存使用量(1.000GiB),不能   分配1.000MiB INFO的大块[服务主题] 2017-06-19   08:52:33,145 GCInspector.java:284 - ConcurrentMarkSweep GC在215ms。   CMS Old Gen:5868761104 - > 3186639968; Par Eden Space:9853592 - >   423279080; Par Survivor Space:83886080 - > 45608368

     

INFO [CompactionExecutor:4037] 2017-06-19 08:57:34,632   NoSpamLogger.java:91 - 达到的最大内存使用量(1.000GiB),不能   分配1.000MiB的块

的debug.log

  

DEBUG [SlabPoolCleaner] 2017-06-19 09:12:59,260   ColumnFamilyStore.java:899 - 排队size_estimates:   78.229MiB(4%)堆上,0.000KiB(0%)堆外DEBUG [PerDiskMemtableFlushWriter_0:6285] 2017-06-19 09:12:59,575   Memtable.java:461 - 写作   Memtable-size_estimates @ 1285230058(21.550MiB序列化字节,150876   ops,on / off-heap限制的4%/ 0%),刷新范围=   (分(-9223372036854775808),最大(9223372036854775807)]

     

DEBUG [MemtableFlushWriter:6230] 2017-06-19 09:12:59,618   ColumnFamilyStore.java:1197 - 刷新到   [BigTableReader(路径=' /var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/mc-21460-big-Data.db')]   (1 sstables,2.340MiB),最大2.340MiB,最小2.340MiB

     

DEBUG [PerDiskMemtableFlushWriter_0:6285] 2017-06-19 09:12:59,895   Memtable.java:490 - 完成冲洗   /var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/mc-21461-big-Data.db   (15.813MiB)用于commitlog位置   CommitLogPosition(segmentId = 1496885272130,position = 32411980)

     

DEBUG [MemtableFlushWriter:6231] 2017-06-19 09:13:00,077   ColumnFamilyStore.java:1197 - 刷新到   [BigTableReader(路径=' /var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/mc-21461-big-Data.db')]   (1 sstables,2.340MiB),最大2.340MiB,最小2.340MiB

     

DEBUG [CompactionExecutor:4043] 2017-06-19 09:13:02,440   CompactionTask.java:255 - 压缩   (0b5956b0-5484-11e7-b5a8-01c062b805b9)4 sstables to   [在/ var / lib中/卡桑德拉/数据/系统/ size_estimates-618f817b005f3678b8a453f3930b8e86 / MC-21458-大,]   等级= 0。 6,349ms,20.780MiB至13.916MiB(原始的约66%)。   读吞吐量= 3.273MiB / s,写吞吐量= 2.192MiB / s,行   吞吐量= ~131,062 / s。 12个分区合并为5.分区   合并计数为{2:4,4:1,}

     

DEBUG [CompactionExecutor:4043] 2017-06-19 09:13:02,440   CompactionTask.java:155 - 压缩   (0f221e81-5484-11e7-b5a8-01c062b805b9)   [/var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/mc-21461-big-Data.db:level=0,   /var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/mc-21460-big-Data.db:level=0,   /var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/mc-21459-big-Data.db:level=0,   /var/lib/cassandra/data/system/size_estimates-618f817b005f3678b8a453f3930b8e86/mc-21458-big-Data.db:level=0,   ]

     

DEBUG [CompactionExecutor:4043] 2017-06-19 09:13:08,453   CompactionTask.java:255 - 压缩   (0f221e81-5484-11e7-b5a8-01c062b805b9)4 sstables to   [在/ var / lib中/卡桑德拉/数据/系统/ size_estimates-618f817b005f3678b8a453f3930b8e86 / MC-21462-大,]   等级= 0。 6,012ms,20.775MiB至13.923MiB(原始的约67%)。   读吞吐量= 3.455MiB / s,写吞吐量= 2.316MiB / s,行   吞吐量= ~131,059 / s。 8个分区合并为5.分区   合并计数为{1:4,4:1,}

任何建议都会有所帮助和赞赏。

1 个答案:

答案 0 :(得分:1)

内存不足(日志的第一行)。很奇怪这是一个信息,而不是一个警告。当您内存不足时,系统可能仍然健康,但无法接受任何新连接。添加更多内存,直到你有足够的运行稳定。