无法从cassandra中的大分区中提取数据

时间:2016-07-25 14:32:43

标签: cassandra cassandra-2.0 datastax-enterprise cassandra-2.1

我在一段时间后意识到我的Cassandra Data模型中的分区大小不正确。我设计了我的主键,只创建了一个分区。所以,我在所有查询中都获得了读取超时。我已经通过创建一个新的列系列解决了这个问题,但现在问题是我如何从这个分区恢复数据任何帮助将不胜感激。我有大约40GB的分区,如下所示:

38G     ./eventdata-7b5004e02bdc11e69ebb19dbbcf6d4b0
55M     ./eventdata-8f8317a01e5711e69ebb19dbbcf6d4b0
2.4M    ./eventdata-3c20dd401cf911e69ebb19dbbcf6d4b0
88K     ./eventdata-1866015012a911e69ebb19dbbcf6d4b0

所以,要读取我使用sstable2json在json中转换它的数据,但它给出了这个错误:

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.nio.HeapByteBuffer.duplicate(HeapByteBuffer.java:107)
        at org.apache.cassandra.db.composites.AbstractCType.sliceBytes(AbstractCType.java:369)
        at org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:101)
        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
        at org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:194)
        at org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:174)
        at org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:129)
        at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:257)
        at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:231)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:211)
        at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:332)
        at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:369)
        at org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:382)
        at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:467)
ERROR 14:02:13 LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@2835df6c) to class org.apache.cassandra.io.sstable.S
STableReader$InstanceTidier@1560150690:/mnt/eventdata/eventdata-7b5004e02bdc11e69ebb19dbbcf6d4b0/./cw-eventdata-ka-2839 was not released before the reference was garbage collected

然后我试图使用sstablesplit实用程序拆分sstable,但我又得到了这个错误。我在一个拥有8GB RAM和8GB交换空间的节点上运行Cassandra 2.1.12:

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:375)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:352)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:331)
        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
        at org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:193)
        at org.apache.cassandra.io.sstable.IndexHelper$IndexInfo$Serializer.deserialize(IndexHelper.java:174)
        at org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:129)
        at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:257)
        at org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.computeNext(SSTableScanner.java:231)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.io.sstable.SSTableScanner.hasNext(SSTableScanner.java:211)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:181)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
        at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
        at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
        at org.apache.cassandra.db.compaction.SSTableSplitter.split(SSTableSplitter.java:38)
        at org.apache.cassandra.tools.StandaloneSplitter.main(StandaloneSplitter.java:153)

0 个答案:

没有答案