当我们向Cassandra发送大量请求时,我们有时会收到
ERROR [ReadRepairStage:12] 2018-05-07 10:35:19,056 CassandraDaemon.java:228 - Exception in thread Thread[ReadRepairStage:12,5,main]
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
at org.apache.cassandra.service.DataResolver$RepairMergeListener.close(DataResolver.java:199) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.close(UnfilteredPartitionIterators.java:175) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:92) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.service.DataResolver.compareResponses(DataResolver.java:76) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.service.AsyncRepairCallback$1.runMayThrow(AsyncRepairCallback.java:50) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.11.1.jar:3.11.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_152]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_152]
at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) ~[apache-cassandra-3.11.1.jar:3.11.1]
at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_152]
知道造成这种情况的原因是什么? 我能提供的任何遗漏信息?
我也看到了
INFO [ReadStage-14] 2018-05-07 10:22:58,869 NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
我怀疑这可能有助于引发问题? 不知道。
更新: 我们有多个节点和多个实例(6个节点每个节点是4个实例' s) 我们也试图调整GC,因为我们怀疑它会导致缓慢