在我的设置中,由于电源中断,我的VM在重新启动所有虚拟机后都处于关闭状态 我无法启动Hbase,所有其他服务可以重新启动/停止/启动但在UI中用于CM 在hbase ACTIONS选项卡中,所有选项都被隐藏了..
以下是我的hbase主日志,现在处于退出模式
6:29:05.880 AM ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
ZooKeeper exists failed after 3 retries
6:29:05.880 AM WARN org.apache.hadoop.hbase.zookeeper.ZKUtil
hconnection Unable to set watcher on znode /hbase/master
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:420)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:76)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:621)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:684)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
at org.apache.hadoop.hbase.master.HMasterCommandLine.stopMaster(HMasterCommandLine.java:175)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:106)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1962)
6:29:05.880 AM ERROR org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher
hconnection Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:420)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:76)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:621)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:684)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
at org.apache.hadoop.hbase.master.HMasterCommandLine.stopMaster(HMasterCommandLine.java:175)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:106)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1962)
6:29:05.881 AM INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation
This client just lost it's session with ZooKeeper, will automatically reconnect when needed.
6:29:06.880 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderaalpha.alp.com/10.0.0.13:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:06.882 AM WARN org.apache.zookeeper.ClientCnxn
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
6:29:06.983 AM WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
6:29:06.983 AM INFO org.apache.hadoop.hbase.util.RetryCounter
Sleeping 2000ms before retry #1...
6:29:06.983 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderadn1.alp.com/10.0.0.14:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:06.984 AM WARN org.apache.zookeeper.ClientCnxn
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
6:29:07.084 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderadn3.alp.com/10.0.0.16:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:27.103 AM INFO org.apache.zookeeper.ClientCnxn
Client session timed out, have not heard from server in 20019ms for sessionid 0x0, closing socket connection and attempting reconnect
6:29:27.204 AM WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
6:29:27.204 AM INFO org.apache.hadoop.hbase.util.RetryCounter
Sleeping 4000ms before retry #2...
6:29:28.204 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderaalpha.alp.com/10.0.0.13:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:28.206 AM WARN org.apache.zookeeper.ClientCnxn
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
6:29:28.306 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderadn1.alp.com/10.0.0.14:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:28.307 AM WARN org.apache.zookeeper.ClientCnxn
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
6:29:28.408 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderadn3.alp.com/10.0.0.16:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:48.425 AM INFO org.apache.zookeeper.ClientCnxn
Client session timed out, have not heard from server in 20017ms for sessionid 0x0, closing socket connection and attempting reconnect
6:29:48.526 AM WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
6:29:48.526 AM INFO org.apache.hadoop.hbase.util.RetryCounter
Sleeping 8000ms before retry #3...
6:29:49.526 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderaalpha.alp.com/10.0.0.13:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:49.529 AM WARN org.apache.zookeeper.ClientCnxn
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
6:29:49.630 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderadn1.alp.com/10.0.0.14:2181. Will not attempt to authenticate using SASL (unknown error)
6:29:49.630 AM WARN org.apache.zookeeper.ClientCnxn
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
6:29:49.731 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server cderadn3.alp.com/10.0.0.16:2181. Will not attempt to authenticate using SASL (unknown error)
6:30:09.744 AM INFO org.apache.zookeeper.ClientCnxn
Client session timed out, have not heard from server in 20013ms for sessionid 0x0, closing socket connection and attempting reconnect
6:30:09.845 AM WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server
6:30:09.845 AM ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
ZooKeeper exists failed after 3 retries
感谢您的关注......
答案 0 :(得分:0)
从日志中我们可以看到它无法连接ZooKeeper服务器。您可能需要启动ZooKeeper服务并尝试重新启动HBase
答案 1 :(得分:0)
正如zhutoulala之前所说,我也会建议,如果它仍然不起作用,你应该删除HBase的znode:
使用运行HBase服务的其中一台服务器上的'hbase zkcli'命令。进入ZooKeeper CLI后,您可以运行'rmr / hbase'来删除znode。