电源故障后,某些查询会抛出Cassandra EOFException

时间:2013-04-09 09:19:25

标签: cassandra

  `INFO 11:44:29,874 Listening for thrift clients...
ERROR 11:47:01,471 Exception in thread Thread[ReadStage:33,5,main]
java.lang.RuntimeException: org.apache.cassandra.io.sstable.CorruptSSTableExcept
ion: java.io.EOFException: EOF after 2 bytes out of 29545
        at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(Stora
geProxy.java:1582)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFE
xception: EOF after 2 bytes out of 29545
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(Simpl
eSliceReader.java:89)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createRea
der(SSTableSliceIterator.java:68)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SS
TableSliceIterator.java:44)
        at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnItera
tor(SliceQueryFilter.java:101)
        at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(Q
ueryFilter.java:68)
        at org.apache.cassandra.db.CollationController.collectAllData(CollationC
ontroller.java:275)
        at org.apache.cassandra.db.CollationController.getTopLevelColumns(Collat
ionController.java:65)
        at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFa
milyStore.java:1363)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamil
yStore.java:1220)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamil
yStore.java:1132)
        at org.apache.cassandra.db.Table.getRow(Table.java:355)
        at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadComm
and.java:70)
        at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThr
ow(StorageProxy.java:1052)
        at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(Stora
geProxy.java:1578)
        ... 3 more
Caused by: java.io.EOFException: EOF after 2 bytes out of 29545
        at org.apache.cassandra.io.util.FileUtils.skipBytesFully(FileUtils.java:
350)
        at org.apache.cassandra.utils.ByteBufferUtil.skipShortLength(ByteBufferU
til.java:382)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(Simpl
eSliceReader.java:71)
        ... 16 more
ERROR 11:47:12,900 Exception in thread Thread[ReadStage:35,5,main]
java.lang.RuntimeException: org.apache.cassandra.io.sstable.CorruptSSTableExcept
ion: java.io.EOFException: EOF after 2 bytes out of 29545
        at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(Stora
geProxy.java:1582)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFE
xception: EOF after 2 bytes out of 29545
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(Simpl
eSliceReader.java:89)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createRea
der(SSTableSliceIterator.java:68)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SS
TableSliceIterator.java:44)
        at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnItera
tor(SliceQueryFilter.java:101)
        at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(Q
ueryFilter.java:68)
        at org.apache.cassandra.db.CollationController.collectAllData(CollationC
ontroller.java:275)
        at org.apache.cassandra.db.CollationController.getTopLevelColumns(Collat
ionController.java:65)
        at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFa
milyStore.java:1363)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamil
yStore.java:1220)
        at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamil
yStore.java:1132)
        at org.apache.cassandra.db.Table.getRow(Table.java:355)
        at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadComm
and.java:70)
        at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThr
ow(StorageProxy.java:1052)
        at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(Stora
geProxy.java:1578)
        ... 3 more
Caused by: java.io.EOFException: EOF after 2 bytes out of 29545
        at org.apache.cassandra.io.util.FileUtils.skipBytesFully(FileUtils.java:
350)
        at org.apache.cassandra.utils.ByteBufferUtil.skipShortLength(ByteBufferU
til.java:382)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(Simpl
eSliceReader.java:71)
        ... 16 more`

在赫克托尔,我得到一个HTimedOutException,我假设是因为卡桑德拉的SSTable腐败,因为它在电源故障之前运作良好。我该如何解决?这也是开发数据库,​​我之前遇到过这个问题并最终删除了一些列系列,但我不希望这是每次断电或系统崩溃时唯一的解决方案。我正在使用cassandra 1.2.2

2 个答案:

答案 0 :(得分:4)

您可以删除损坏的SSTable,然后运行修复以从副本中恢复数据,而不是删除整个CF.问题是错误不会在日志中打印文件名,因此很难找出哪个文件已损坏。您可以通过在调试模式下运行Cassandra来获取它(编辑log4j-server.properties)。见另一篇SO帖子:

Error messages while starting Cassandra node

他们是如何做到的。

答案 1 :(得分:0)

首先要尝试的是nodetool scrub这个cf(例如,参见this post for v.1.2)。