在Hbase shell中运行命令时,“节点/ hbase不在ZooKeeper中”

时间:2017-01-03 22:35:54

标签: hadoop hbase apache-zookeeper

我正在尝试在独立模式下安装HBase v1.1.2(使用Hadoop v2.7.2),并尽可能地遵循Apache HBase docs,但配置似乎有点棘手。不确定这是否与我使用自制软件安装HBase相反,而不是通过其中一个镜像站点直接下载。我的目标是达到我可以做的事情,比如创建表,列出现有表等等。

我安装了Java 1.7.0,并运行了“brew install hbase”来安装hbase。在Homebrew的hbase目录中,我打开了/libexec/conf/hbase-env.sh,我在第27行看到了以下内容:

export JAVA_HOME="$(/usr/libexec/java_home)"

当我在命令行中执行/usr/libexec/java_home时,它会输出/Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home。所以似乎设置了一个有效的$ JAVA_HOME env变量。

在我的/libexec/conf/hbase-site.xml配置文件中,我设置了以下键/值对:

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>file:///home/richiethomas/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/home/richiethomas/zookeeper</value>
  </property>
</configuration>

换句话说,我已经制作了HBase文档中显示的配置的精确副本,只将'testuser'替换为'richiethomas',因为这是我在命令行中使用的用户名。

然后在Homebrew hbase目录中运行`./bin/start-hbase.sh'。我看到以下内容:

starting master, logging to /usr/local/Cellar/hbase/1.1.2/libexec/bin/../logs/hbase-richiethomas-master-MacBook-Pro-5.local.out

然后,我运行hbase shell,我看到以下内容:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hbase/1.1.2/libexec/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hadoop/2.7.2/libexec/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-01-03 17:27:52,056 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.1.2, rcc2b70cf03e3378800661ec5cab11eb43fafe0fc, Wed Aug 26 20:11:27 PDT 2015

>>

执行此操作后,在新的CLI选项卡中cat显示上述路径中的日志(/usr/local/Cellar/hbase/1.1.2/libexec/bin/../logs/hbase-richiethomas-master-MacBook-Pro-5.local.out),我看到以下内容:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hbase/1.1.2/libexec/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hadoop/2.7.2/libexec/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

然后,回到我的HBase shell中,运行以下命令以查看是否一切正常,我看到以下内容:

>> list
TABLE                                                                                                                                                                            
2017-01-03 17:28:05,108 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:05,320 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:05,625 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:06,128 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:07,134 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:09,144 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:13,158 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.

ERROR: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.

Here is some help for this command:
List all tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:

  hbase> list
  hbase> list 'abc.*'
  hbase> list 'ns:abc.*'
  hbase> list 'ns:.*'

根据Apache文档:

  

您无需创建HBase数据目录。 HBase会这样做   为了你。如果您创建目录,HBase将尝试执行此操作   迁移,这不是你想要的。

我认为这意味着我不需要在hbase-site.xml中设置以下属性,特别是因为'hbase'是'zookeeper.znode.parent'的默认值:

<property>
    <name>zookeeper.znode.parent</name>
    <value>/hbase</value>
</property>

jps的输出如下:

32146 ZooKeeperMain
1364 QuorumPeerMain
83568 
34120 Jps

我的zoo.cfg文件位于/usr/local/etc/zookeeper/zoo.cfg,如下所示:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/usr/local/var/run/zookeeper/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

我试过的解决方案:

我尝试用zookeeper.property.dataDir替换hbase-site.xml文件中/usr/local/var/run/zookeeper/data的值,但这并没有解决问题。

II。

当我在命令行中运行zkcli并运行ls /时,我只看到zookeeper,没有hbase目录。我尝试在zookeeper shell中运行以下命令:

create /hbase my_data

然后我看到以下内容:

[zk: localhost:2181(CONNECTED) 4] ls /
[hbase, zookeeper]

然后当我退出并重新运行HBase shell并尝试list时,我看到以下内容:

>> list
TABLE                                                                                                                                                                            

ERROR: Can't get master address from ZooKeeper; znode data == null

III。

根据this Quora article,HBase 1.1.2仅支持Hadoop到2.5.x版本,因此我运行brew uninstall hadoop后跟brew install hadoop25,运行停止并启动hbase脚本,以及重新启动了hbase shell。仍然没有运气。

IV

我尝试卸载Homebrew版本并从Apache镜像站点下载HBase,并在hbase-site.xml中重复配置过程,但我得到了同样的错误(The node /hbase is not in ZooKeeper. It should have been written by the master.)。

我通过Homebrew卸载/重新安装了HBase,并按照“警告”部分中的说明进行操作:

==> Caveats
To have launchd start hbase now and restart at login:
  brew services start hbase
Or, if you don't want/need a background service you can just run:
  /usr/local/opt/hbase/bin/start-hbase.sh

我注意到在运行上述命令后,hbase-site.xml文件中填充了hbase.rootdirhbase.zookeeper.property.dataDir的具有合理外观的值(具体来说,file:///usr/local/var/hbase和{{ 1}},分别)。我还注意到,当我运行/usr/local/var/zookeeper,然后运行brew services start hbase时,我没有看到“HMaster”进程正在运行,但是当我运行jps时,我确实看到了HMaster。但是,我尝试启动/usr/local/opt/hbase/bin/start-hbase.sh并输入hbase shell,我收到了与之前相同的错误(status)。

VI。

我已经验证我的Hadoop安装运行正常:

The node /hbase is not in ZooKeeper.

我的HBase配置似乎有问题,但我不确定还有什么可以尝试。

1 个答案:

答案 0 :(得分:6)

由于您要设置独立的hbase,因此无需构建hdfszk

问题的原因在于,hbase将尝试启动内部zk服务,但由于已经有一个java进程QuorumPeerMain占用端口2181hbase将无法启动。如果hbase成功启动,则会有一个名为HMaster

的java进程

要验证这一点,请检查日志hbase-richiethomas-master-MacBook-Pro-5.local.out的最后一行,您可能会看到如下错误消息:

Could not start ZK at requested port of 2181.  ZK was started at port: 2182.  Aborting as clients (e.g. shell) will not be able to find this ZK quorum.

所以,解决方案是:

  1. 杀死现有的QuorumPeerMain
  2. 再次运行start-hbase.sh