Cassandra UNREACHABLE节点

时间:2017-07-12 13:41:03

标签: cassandra-3.0

我有Cassandra v3.9,3节点集群。复制因子2

10.0.0.11,10.0.0.12,10.0.0.13

10.0.0.11,10.0.0.12是种子节点

在/etc/cassandra/conf/debug.log中出现以下错误的可能原因

错误是

DEBUG [RMI TCP连接(174)-127.0.0.1] 2017-07-12 04:47:49,002 StorageProxy.java:2254 - 主机不同意。没有得到每个人的回复:10.0.0.13

UPDATE1:

在出现错误时,这里有来自所有服务器的一些统计信息

[user1@ip-10-0-0-11 ~]$ nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load       Tokens       Owns (effective)  Host ID                               Rack
UN  10.0.0.12  2.55 MiB   256          67.2%             83a68750-2238-4a6e-87be-03a3d7246824  rack1
UN  10.0.0.11  1.78 GiB   256          70.6%             052fda9d-0474-4dfb-b2f8-0c5cbec15266  rack1
UN  10.0.0.13  1.78 GiB   256          62.2%             86438dc9-77e0-43b2-a672-5b2e7cf216bf  rack1

[user1@ip-10-0-0-11 ~]$ nodetool describecluster
Cluster Information:
    Name: PiedmontCluster
    Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
    Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
    Schema versions:
        3c8d9e82-c688-3d16-a3e9-b84894168283: [10.0.0.12, 10.0.0.11]

        UNREACHABLE: [10.0.0.13]


[pnm@ip-10-0-0-13 ~]$ nodetool status
    Datacenter: datacenter1
    =======================
    Status=Up/Down
    |/ State=Normal/Leaving/Joining/Moving
    --  Address     Load       Tokens       Owns (effective)  Host ID                               Rack
    UN  10.0.0.12  2.55 MiB   256          67.2%             83a68750-2238-4a6e-87be-03a3d7246824  rack1
    UN  10.0.0.11  1.78 GiB   256          70.6%             052fda9d-0474-4dfb-b2f8-0c5cbec15266  rack1
    UN  10.0.0.13  1.78 GiB   256          62.2%             86438dc9-77e0-43b2-a672-5b2e7cf216bf  rack1

    [pnm@ip-10-0-0-13 ~]$ nodetool describecluster
        Cluster Information:
        Name: PiedmontCluster
        Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
        Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
        Schema versions:
            3c8d9e82-c688-3d16-a3e9-b84894168283: [10.0.0.12, 10.0.0.13]

        UNREACHABLE: [10.0.0.11]


[user1@ip-10-0-0-12 ~]$ nodetool status
    Datacenter: datacenter1
    =======================
    Status=Up/Down
    |/ State=Normal/Leaving/Joining/Moving
    --  Address     Load       Tokens       Owns (effective)  Host ID                               Rack
    UN  10.0.0.12  2.55 MiB   256          67.2%             83a68750-2238-4a6e-87be-03a3d7246824  rack1
    UN  10.0.0.11  1.78 GiB   256          70.6%             052fda9d-0474-4dfb-b2f8-0c5cbec15266  rack1
    UN  10.0.0.13  1.78 GiB   256          62.2%             86438dc9-77e0-43b2-a672-5b2e7cf216bf  rack1

    [user1@ip-10-0-0-12 ~]$ nodetool describecluster
        Cluster Information:
        Name: PiedmontCluster
        Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
            Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
            Schema versions:
                3c8d9e82-c688-3d16-a3e9-b84894168283: [10.0.0.12, 10.0.0.11, 10.0.0.13]

上述错误发生在10.0.0.11的/var/log/cassandra/debug.log中

10.0.0.13上/var/log/cassandra/debug.php中的错误

    DEBUG [RMI TCP Connection(4)-127.0.0.1] 2017-07-13 02:31:23,846       StorageProxy.java:2254 - Hosts not in agreement. Didn't get a response from everybody: 10.0.0.11
    ERROR [MessagingService-Incoming-/10.0.0.11] 2017-07-13 02:35:04,982 CassandraDaemon.java:226 - Exception in thread Thread[MessagingService-Incoming-/10.0.0.11,5,main]
    java.lang.ArrayIndexOutOfBoundsException: 4
        at  org.apache.cassandra.db.filter.AbstractClusteringIndexFilter$FilterSerializer.deserialize(AbstractClusteringIndexFilter.java:74) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.db.SinglePartitionReadCommand$Deserializer.deserialize(SinglePartitionReadCommand.java:1041) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:696) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:626) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.io.ForwardingVersionedSerializer.deserialize(ForwardingVersionedSerializer.java:50) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178) ~[apache-cassandra-3.9.0.jar:3.9.0]
        at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92) ~[apache-cassandra-3.9.0.jar:3.9.0]

10.0.0.12上的/var/log/cassandra/debug.php没有错误

记住10.0.0.11& 10.0.0.12是种子节点

由于

0 个答案:

没有答案