Nodetool修复cassandra 3.9集群抱怨死节点的错误

时间:2018-02-19 12:45:56

标签: cassandra

我有一个cassandra 3.9群集。我从群集中的一个节点启动了修复。修复无处可去。我看到启动节点上的日志填充了这样的错误。

ERROR [GossipTasks:1] 2018-02-16 23:27:36,949 RepairSession.java:347 - [repair #cadf6f11-1342-11e8-8d73-6767c6890f70] session completed with the following error
java.io.IOException: Endpoint /**.**.**.52 died
        at org.apache.cassandra.repair.RepairSession.convict(RepairSession.java:346) ~[apache-cassandra-3.9.jar:3.9]
        at org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:306) [apache-cassandra-3.9.jar:3.9]
        at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:782) [apache-cassandra-3.9.jar:3.9]
        at org.apache.cassandra.gms.Gossiper.access$800(Gossiper.java:66) [apache-cassandra-3.9.jar:3.9]
        at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:181) [apache-cassandra-3.9.jar:3.9]
        at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) [apache-cassandra-3.9.jar:3.9]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_91]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_91]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_91]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_91]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]

另一方面,如果我查看声称死亡的节点的日志,我会看到3种症状中的一种。

  1. 节点声称已成功发送请求的节点 merkle tree over。
  2. 节点没有任何修复会话的痕迹,因此似乎没有收到任何修复请求。
  3. 节点显示如下异常。

    ERROR [ValidationExecutor:3] 2018-02-16 23:29:06,548 Validator.java:261 - 无法创建一个merkle树[修复#cac2bf50-1342-11e8-8d73-6767c6890f70 on somekeyspace / sometable,[( -3531087107126953137,-3495591103116433105],(1424707151780052485,1425479237398192865],(-3533012126945497873,-3531087107126953137],(1425479237398192865,1429220273719165251],(-4991682772598302168,-4984938905452900436],(-7686750611814623539,-7685228552629222537],(7554301216433235881,7559623046999138658],( 334796420453180909,342318143371667659],( - 3538876023288368831,-3533012126945497873],(140914567521922418,1424707151780052485),(5391546013321073004,5393284101537339558),(590921410556013711,593440512568877190]]],/ .**.43(见日志详情)

    ERROR [ValidationExecutor:3] 2018-02-16 23:29:06,549 CassandraDaemon.java:226 - 线程中的异常线程[ValidationExecutor:3,1,main] java.lang.RuntimeException:id = c8bf7540-1342-11e8-8d73-6767c6890f70的父修复会话失败。         在org.apache.cassandra.service.ActiveRepairService.getParentRepairSession(ActiveRepairService.java:377)〜[apache-cassandra-3.9.jar:3.9]         在org.apache.cassandra.db.compaction.CompactionManager.getSSTablesToValidate(CompactionManager.java:1313)〜[apache-cassandra-3.9.jar:3.9]         在org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1222)〜[apache-cassandra-3.9.jar:3.9]         在org.apache.cassandra.db.compaction.CompactionManager.access $ 700(CompactionManager.java:81)〜[apache-cassandra-3.9.jar:3.9]         在org.apache.cassandra.db.compaction.CompactionManager $ 11.call(CompactionManager.java:844)〜[apache-cassandra-3.9.jar:3.9]         在java.util.concurrent.FutureTask.run(FutureTask.java:266)〜[na:1.8.0_91]         在java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)〜[na:1.8.0_91]         at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617)[na:1.8.0_91]         在java.lang.Thread.run(Thread.java:745)[na:1.8.0_91]

  4. 这是一个已知问题吗?

0 个答案:

没有答案