网络出现问题后,我们的hazelcast集群以两个领导者结束。将来如何避免这种情况?
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
09:57:15 INFO TcpIpConnection:49 - [HOST_1]:PORT [dev] [3.5.5] Connection [Address[HOST_2]:PORT] lost. Reason: Socket explicitly closed
09:57:15 WARN WriteHandler:49 - [HOST_1]:PORT [dev] [3.5.5] hz._hzInstance_1_dev.IO.thread-out-2 Closing socket to endpoint Address[HOST_2]:PORT, Cause:java.nio.channels.CancelledKeyException
java.nio.channels.CancelledKeyException
09:57:15 INFO ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] Removing Member [HOST_2]:PORT
09:57:15 INFO ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5]
Members [2] {
Member [HOST_3]:PORT
Member [HOST_1]:PORT this
}
09:57:16 INFO SocketAcceptor:49 - [HOST_1]:PORT [dev] [3.5.5] Accepting socket connection from /HOST_2:54380
09:57:16 INFO TcpIpConnectionManager:49 - [HOST_1]:PORT [dev] [3.5.5] Established socket connection between /HOST_1:PORT
09:57:23 WARN MasterConfirmationOperation:49 - [HOST_1]:PORT [dev] [3.5.5] MasterConfirmation has been received from Address[HOST_2]:PORT, but it is not a member of this cluster!
09:57:23 INFO TcpIpConnection:49 - [HOST_1]:PORT [dev] [3.5.5] Connection [Address[HOST_2]:PORT] lost. Reason: java.io.EOFException[Remote socket closed!]
09:57:23 WARN ReadHandler:49 - [HOST_1]:PORT [dev] [3.5.5] hz._hzInstance_1_dev.IO.thread-in-0 Closing socket to endpoint Address[HOST_2]:PORT, Cause:java.io.EOFException: Remote socket closed!
09:58:34 INFO SocketAcceptor:49 - [HOST_1]:PORT [dev] [3.5.5] Accepting socket connection from /HOST_2:54009
09:58:34 INFO TcpIpConnectionManager:49 - [HOST_1]:PORT [dev] [3.5.5] Established socket connection between /HOST_1:PORT
09:58:36 INFO TcpIpConnection:49 - [HOST_1]:PORT [dev] [3.5.5] Connection [Address[HOST_2]:PORT] lost. Reason: java.io.EOFException[Remote socket closed!]
09:58:36 WARN ReadHandler:49 - [HOST_1]:PORT [dev] [3.5.5] hz._hzInstance_1_dev.IO.thread-in-1 Closing socket to endpoint Address[HOST_2]:PORT, Cause:java.io.EOFException: Remote socket closed!
09:58:36 INFO SocketAcceptor:49 - [HOST_1]:PORT [dev] [3.5.5] Accepting socket connection from /HOST_2:46357
09:58:36 INFO TcpIpConnectionManager:49 - [HOST_1]:PORT [dev] [3.5.5] Established socket connection between /HOST_1:PORT
10:04:11 INFO TcpIpConnection:49 - [HOST_1]:PORT [dev] [3.5.5] Connection [Address[HOST_2]:PORT] lost. Reason: java.io.EOFException[Remote socket closed!]
10:04:11 WARN ReadHandler:49 - [HOST_1]:PORT [dev] [3.5.5] hz._hzInstance_1_dev.IO.thread-in-2 Closing socket to endpoint Address[HOST_2]:PORT, Cause:java.io.EOFException: Remote socket closed!
10:05:06 INFO TcpIpConnection:49 - [HOST_1]:PORT [dev] [3.5.5] Connection [Address[HOST_3]:PORT] lost. Reason: java.io.EOFException[Remote socket closed!]
10:05:06 WARN ReadHandler:49 - [HOST_1]:PORT [dev] [3.5.5] hz._hzInstance_1_dev.IO.thread-in-1 Closing socket to endpoint Address[HOST_3]:PORT, Cause:java.io.EOFException: Remote socket closed!
10:05:06 INFO SocketConnector:49 - [HOST_1]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
10:05:11 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:16 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:21 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:24 INFO some_log:159 - Is leader: false
10:05:26 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:31 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:36 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:41 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:46 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:51 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:05:56 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:06:01 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
10:06:06 INFO SocketAcceptor:49 - [HOST_1]:PORT [dev] [3.5.5] Accepting socket connection from /HOST_3:51628
10:06:06 INFO TcpIpConnectionManager:49 - [HOST_1]:PORT [dev] [3.5.5] Established socket connection between /HOST_1:PORT
10:06:08 INFO TcpIpConnection:49 - [HOST_1]:PORT [dev] [3.5.5] Connection [Address[HOST_3]:PORT] lost. Reason: java.io.EOFException[Remote socket closed!]
10:06:08 WARN ReadHandler:49 - [HOST_1]:PORT [dev] [3.5.5] hz._hzInstance_1_dev.IO.thread-in-0 Closing socket to endpoint Address[HOST_3]:PORT, Cause:java.io.EOFException: Remote socket closed!
10:06:08 INFO SocketAcceptor:49 - [HOST_1]:PORT [dev] [3.5.5] Accepting socket connection from /HOST_3:57310
10:06:08 INFO TcpIpConnectionManager:49 - [HOST_1]:PORT [dev] [3.5.5] Established socket connection between /HOST_1:PORT
10:06:09 INFO SocketConnector:49 - [HOST_1]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: SocketException[Connection timed out to address /HOST_3:PORT]
10:06:09 WARN ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] New join request has been received from an existing endpoint! => Member [HOST_3]:PORT Removing old member and processing join request...
10:06:09 INFO ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] Master Address[HOST_3]:PORT left the cluster. Assigning new master Member [HOST_1]:PORT this
10:06:09 INFO ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5] Removing Member [HOST_3]:PORT
10:06:09 INFO ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5]
Members [1] {
Member [HOST_1]:PORT this
}
10:06:17 INFO ClusterService:49 - [HOST_1]:PORT [dev] [3.5.5]
Members [2] {
Member [HOST_1]:PORT this
Member [HOST_3]:PORT
}
10:06:24 INFO some_log:159 - Is leader: true
09:56:53 INFO some_log:159 - Is leader: false
09:57:03 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Removing Member [HOST_3]:PORT because it has not sent any heartbeats for 300000 ms.
09:57:03 INFO ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Master Address[HOST_3]:PORT left the cluster. Assigning new master Member [HOST_1]:PORT
09:57:04 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] This node does not have a connection to Member [HOST_3]:PORT
09:57:04 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
09:57:05 INFO TcpIpConnection:49 - [HOST_2]:PORT [dev] [3.5.5] Connection [Address[HOST_3]:PORT] lost. Reason: Socket explicitly closed
09:57:05 INFO ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Removing Member [HOST_3]:PORT
09:57:05 INFO ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5]
Members [2] {
Member [HOST_1]:PORT
Member [HOST_2]:PORT this
}
09:57:15 INFO TcpIpConnection:49 - [HOST_2]:PORT [dev] [3.5.5] Connection [Address[HOST_1]:PORT] lost. Reason: java.io.EOFException[Remote socket closed!]
09:57:15 WARN ReadHandler:49 - [HOST_2]:PORT [dev] [3.5.5] hz._hzInstance_1_dev.IO.thread-in-0 Closing socket to endpoint Address[HOST_1]:PORT, Cause:java.io.EOFException: Remote socket closed!
09:57:16 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_1:PORT, timeout: 0, bind-any: true
09:57:16 INFO TcpIpConnectionManager:49 - [HOST_2]:PORT [dev] [3.5.5] Established socket connection between /HOST_2:54380
09:57:23 INFO ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Master Address[HOST_1]:PORT left the cluster. Assigning new master Member [HOST_2]:PORT this
09:57:23 INFO TcpIpConnection:49 - [HOST_2]:PORT [dev] [3.5.5] Connection [Address[HOST_1]:PORT] lost. Reason: Socket explicitly closed
09:57:23 INFO ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Removing Member [HOST_1]:PORT
09:57:23 INFO ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5]
Members [1] {
Member [HOST_2]:PORT this
}
09:57:53 INFO some_log:159 - Is leader: true
09:57:53 INFO SpringCamelContext:3564 - Route: F2MPollFromDBAndAddToQueueTryAgainMails started and consuming from: Endpoint[timer://name?delay=60000&period=60000]
09:57:53 INFO SpringCamelContext:3564 - Route: pollFromDBAndAddToQueueErrorMails started and consuming from: Endpoint[timer://Mail2FaxAddTryAgain?delay=6000&period=60000]
09:57:53 INFO SpringCamelContext:3564 - Route: notifyCustomersWaitingForFaxSending started and consuming from: Endpoint[timer://name?delay=5000&period=300000]
09:57:53 INFO SpringCamelContext:3564 - Route: mailReceiverRoute started and consuming from: Endpoint[imaps://126.179.4.208:993?consumer.delay=5000&delete=true&dummyTrustManager=true&password=xxxxxx&unseen=false&username=ims%40fax.orange.pl]
09:57:53 INFO SpringCamelContext:3564 - Route: mailReceiverRoute started and consuming from: Endpoint[direct://mailInject]
09:57:53 INFO SpringCamelContext:3564 - Route: mailReceiverRoute started and consuming from: Endpoint[imaps://126.179.4.208:993?consumer.delay=5000&delete=true&dummyTrustManager=true&password=xxxxxx&unseen=false&username=ims%40fax.orange.pl]
09:57:53 INFO SpringCamelContext:3564 - Route: mailReceiverRoute started and consuming from: Endpoint[direct://mailInject]
09:58:07 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: SocketException[Connection timed out to address /HOST_3:PORT]
09:58:34 INFO TcpIpConnectionManager:49 - [HOST_2]:PORT [dev] [3.5.5] Established socket connection between /HOST_2:54009
09:58:35 INFO TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_2]:PORT is merging to Address[HOST_1]:PORT, because : joinRequest.getMemberCount() > currentMemberCount [2 > 1]
09:58:35 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_2]:PORT is merging [tcp/ip] to Address[HOST_1]:PORT
09:58:35 WARN PrepareMergeOperation:49 - [HOST_2]:PORT [dev] [3.5.5] Preparing to merge... Waiting for merge instruction...
09:58:35 WARN MergeClustersOperation:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_2]:PORT is merging to Address[HOST_1]:PORT, because: instructed by master Address[HOST_2]:PORT
09:58:35 INFO LifecycleService:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_2]:PORT is MERGING
09:58:36 INFO TcpIpConnection:49 - [HOST_2]:PORT [dev] [3.5.5] Connection [Address[HOST_1]:PORT] lost. Reason: Socket explicitly closed
09:58:36 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_1:PORT, timeout: 0, bind-any: true
09:58:36 INFO TcpIpConnectionManager:49 - [HOST_2]:PORT [dev] [3.5.5] Established socket connection between /HOST_2:46357
09:58:37 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:37 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
09:58:38 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:39 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:40 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:41 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:42 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:43 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:44 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:45 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:46 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:47 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:48 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:49 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:50 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:51 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:52 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:53 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:54 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:55 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:58:56 WARN ClusterService:49 - [HOST_2]:PORT [dev] [3.5.5] Could not create connection to possible master Address[HOST_3]:PORT
09:59:31 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
09:59:40 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: SocketException[Connection timed out to address /HOST_3:PORT]
09:59:40 INFO TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is added to the blacklist.
09:59:40 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
10:00:06 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:00:41 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:00:43 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: SocketException[Connection timed out to address /HOST_3:PORT]
10:00:43 INFO TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is added to the blacklist.
10:00:43 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
10:01:16 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:01:46 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: SocketException[Connection timed out to address /HOST_3:PORT]
10:01:46 INFO TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is added to the blacklist.
10:01:46 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
10:01:51 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:02:26 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:02:49 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: SocketException[Connection timed out to address /HOST_3:PORT]
10:02:49 INFO TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is added to the blacklist.
10:02:49 WARN TcpIpConnectionMonitor:49 - [HOST_2]:PORT [dev] [3.5.5] Removing connection to endpoint Address[HOST_3]:PORT Cause => java.net.SocketException {Connection timed out to address /HOST_3:PORT}, Error-Count: 5
10:02:49 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
10:03:01 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:03:36 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:03:52 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: SocketException[Connection timed out to address /HOST_3:PORT]
10:03:52 INFO TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is added to the blacklist.
10:03:52 WARN TcpIpConnectionMonitor:49 - [HOST_2]:PORT [dev] [3.5.5] Removing connection to endpoint Address[HOST_3]:PORT Cause => java.net.SocketException {Connection timed out to address /HOST_3:PORT}, Error-Count: 6
10:03:52 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Connecting to /HOST_3:PORT, timeout: 0, bind-any: true
10:04:11 WARN TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Couldn't join to the master : Address[HOST_3]:PORT
10:04:11 ERROR Node:49 - [HOST_2]:PORT [dev] [3.5.5] Could not join cluster in 300000 ms. Shutting down now!
10:04:11 INFO LifecycleService:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_2]:PORT is SHUTTING_DOWN
10:04:11 WARN Node:49 - [HOST_2]:PORT [dev] [3.5.5] Terminating forcefully...
10:04:11 INFO Node:49 - [HOST_2]:PORT [dev] [3.5.5] Shutting down connection manager...
10:04:11 INFO TcpIpConnection:49 - [HOST_2]:PORT [dev] [3.5.5] Connection [Address[HOST_1]:PORT] lost. Reason: Socket explicitly closed
10:04:11 INFO Node:49 - [HOST_2]:PORT [dev] [3.5.5] Shutting down node engine...
10:04:14 INFO NodeExtension:49 - [HOST_2]:PORT [dev] [3.5.5] Destroying node NodeExtension.
10:04:14 INFO Node:49 - [HOST_2]:PORT [dev] [3.5.5] Hazelcast Shutdown is completed in 3325 ms.
10:04:14 INFO LifecycleService:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_2]:PORT is SHUTDOWN
10:04:14 INFO SocketConnector:49 - [HOST_2]:PORT [dev] [3.5.5] Could not connect to: /HOST_3:PORT. Reason: ClosedByInterruptException[null]
10:04:14 INFO TcpIpJoiner:49 - [HOST_2]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is added to the blacklist.
09:57:02 WARN ClusterService:49 - [HOST_3]:PORT [dev] [3.5.5] Removing Member [HOST_2]:PORT because it has not sent any heartbeats for 300000 ms.
09:57:02 INFO TcpIpConnection:49 - [HOST_3]:PORT [dev] [3.5.5] Connection [Address[HOST_2]:PORT] lost. Reason: Socket explicitly closed
09:57:02 INFO ClusterService:49 - [HOST_3]:PORT [dev] [3.5.5] Removing Member [HOST_2]:PORT
09:57:02 INFO ClusterService:49 - [HOST_3]:PORT [dev] [3.5.5]
Members [2] {
Member [HOST_3]:PORT this
Member [HOST_1]:PORT
}
10:04:50 WARN ClusterService:49 - [HOST_3]:PORT [dev] [3.5.5] Removing Member [HOST_1]:PORT because it has not sent any heartbeats for 300000 ms.
10:04:50 INFO TcpIpConnection:49 - [HOST_3]:PORT [dev] [3.5.5] Connection [Address[HOST_1]:PORT] lost. Reason: Socket explicitly closed
10:04:50 INFO ClusterService:49 - [HOST_3]:PORT [dev] [3.5.5] Removing Member [HOST_1]:PORT
10:04:50 INFO ClusterService:49 - [HOST_3]:PORT [dev] [3.5.5]
Members [1] {
Member [HOST_3]:PORT this
}
10:06:06 INFO TcpIpConnectionManager:49 - [HOST_3]:PORT [dev] [3.5.5] Established socket connection between /HOST_3:51628
10:06:08 INFO TcpIpJoiner:49 - [HOST_3]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is merging to Address[HOST_1]:PORT, because : joinRequest.getMemberCount() > currentMemberCount [2 > 1]
10:06:08 WARN TcpIpJoiner:49 - [HOST_3]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is merging [tcp/ip] to Address[HOST_1]:PORT
10:06:08 WARN PrepareMergeOperation:49 - [HOST_3]:PORT [dev] [3.5.5] Preparing to merge... Waiting for merge instruction...
10:06:08 WARN MergeClustersOperation:49 - [HOST_3]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is merging to Address[HOST_1]:PORT, because: instructed by master Address[HOST_3]:PORT
10:06:08 INFO LifecycleService:49 - [HOST_3]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is MERGING
10:06:08 INFO TcpIpConnection:49 - [HOST_3]:PORT [dev] [3.5.5] Connection [Address[HOST_1]:PORT] lost. Reason: Socket explicitly closed
10:06:08 INFO SocketConnector:49 - [HOST_3]:PORT [dev] [3.5.5] Connecting to /HOST_1:PORT, timeout: 0, bind-any: true
10:06:08 INFO TcpIpConnectionManager:49 - [HOST_3]:PORT [dev] [3.5.5] Established socket connection between /HOST_3:57310
10:06:09 INFO some_log:159 - Is leader: true
10:06:17 INFO ClusterService:49 - [HOST_3]:PORT [dev] [3.5.5]
Members [2] {
Member [HOST_1]:PORT
Member [HOST_3]:PORT this
}
10:06:19 INFO LifecycleService:49 - [HOST_3]:PORT [dev] [3.5.5] Address[HOST_3]:PORT is MERGED
答案 0 :(得分:1)
看起来像集群成员一样遭受了网络分裂,并试图从发生的合并操作中恢复。只有在网络恢复后才能检测到分裂。但是Quorum之类的机制可以防止脑裂情况。 https://docs.hazelcast.org//docs/3.11.1/manual/html-single/index.html#network-partitioning