我正在尝试在独立模式下安装HBase v1.1.2(使用Hadoop v2.7.2),并尽可能地遵循Apache HBase docs,但配置似乎有点棘手。不确定这是否与我使用自制软件安装HBase相反,而不是通过其中一个镜像站点直接下载。我的目标是达到我可以做的事情,比如创建表,列出现有表等等。
我安装了Java 1.7.0,并运行了“brew install hbase”来安装hbase。在Homebrew的hbase目录中,我打开了/libexec/conf/hbase-env.sh,我在第27行看到了以下内容:
export JAVA_HOME="$(/usr/libexec/java_home)"
当我在命令行中执行/usr/libexec/java_home
时,它会输出/Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home
。所以似乎设置了一个有效的$ JAVA_HOME env变量。
在我的/libexec/conf/hbase-site.xml
配置文件中,我设置了以下键/值对:
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///home/richiethomas/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/richiethomas/zookeeper</value>
</property>
</configuration>
换句话说,我已经制作了HBase文档中显示的配置的精确副本,只将'testuser'替换为'richiethomas',因为这是我在命令行中使用的用户名。
然后在Homebrew hbase
目录中运行`./bin/start-hbase.sh'。我看到以下内容:
starting master, logging to /usr/local/Cellar/hbase/1.1.2/libexec/bin/../logs/hbase-richiethomas-master-MacBook-Pro-5.local.out
然后,我运行hbase shell
,我看到以下内容:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hbase/1.1.2/libexec/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hadoop/2.7.2/libexec/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-01-03 17:27:52,056 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.1.2, rcc2b70cf03e3378800661ec5cab11eb43fafe0fc, Wed Aug 26 20:11:27 PDT 2015
>>
执行此操作后,在新的CLI选项卡中cat
显示上述路径中的日志(/usr/local/Cellar/hbase/1.1.2/libexec/bin/../logs/hbase-richiethomas-master-MacBook-Pro-5.local.out
),我看到以下内容:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hbase/1.1.2/libexec/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hadoop/2.7.2/libexec/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
然后,回到我的HBase shell中,运行以下命令以查看是否一切正常,我看到以下内容:
>> list
TABLE
2017-01-03 17:28:05,108 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:05,320 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:05,625 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:06,128 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:07,134 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:09,144 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
2017-01-03 17:28:13,158 ERROR [main] client.ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
ERROR: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
Here is some help for this command:
List all tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:
hbase> list
hbase> list 'abc.*'
hbase> list 'ns:abc.*'
hbase> list 'ns:.*'
根据Apache文档:
您无需创建HBase数据目录。 HBase会这样做 为了你。如果您创建目录,HBase将尝试执行此操作 迁移,这不是你想要的。
我认为这意味着我不需要在hbase-site.xml
中设置以下属性,特别是因为'hbase'是'zookeeper.znode.parent'的默认值:
<property>
<name>zookeeper.znode.parent</name>
<value>/hbase</value>
</property>
jps
的输出如下:
32146 ZooKeeperMain
1364 QuorumPeerMain
83568
34120 Jps
我的zoo.cfg
文件位于/usr/local/etc/zookeeper/zoo.cfg
,如下所示:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/usr/local/var/run/zookeeper/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
我试过的解决方案:
我
我尝试用zookeeper.property.dataDir
替换hbase-site.xml
文件中/usr/local/var/run/zookeeper/data
的值,但这并没有解决问题。
II。
当我在命令行中运行zkcli
并运行ls /
时,我只看到zookeeper
,没有hbase
目录。我尝试在zookeeper shell中运行以下命令:
create /hbase my_data
然后我看到以下内容:
[zk: localhost:2181(CONNECTED) 4] ls /
[hbase, zookeeper]
然后当我退出并重新运行HBase shell并尝试list
时,我看到以下内容:
>> list
TABLE
ERROR: Can't get master address from ZooKeeper; znode data == null
III。
根据this Quora article,HBase 1.1.2仅支持Hadoop到2.5.x版本,因此我运行brew uninstall hadoop
后跟brew install hadoop25
,运行停止并启动hbase脚本,以及重新启动了hbase shell。仍然没有运气。
IV
我尝试卸载Homebrew版本并从Apache镜像站点下载HBase,并在hbase-site.xml中重复配置过程,但我得到了同样的错误(The node /hbase is not in ZooKeeper. It should have been written by the master.
)。
诉
我通过Homebrew卸载/重新安装了HBase,并按照“警告”部分中的说明进行操作:
==> Caveats
To have launchd start hbase now and restart at login:
brew services start hbase
Or, if you don't want/need a background service you can just run:
/usr/local/opt/hbase/bin/start-hbase.sh
我注意到在运行上述命令后,hbase-site.xml
文件中填充了hbase.rootdir
和hbase.zookeeper.property.dataDir
的具有合理外观的值(具体来说,file:///usr/local/var/hbase
和{{ 1}},分别)。我还注意到,当我运行/usr/local/var/zookeeper
,然后运行brew services start hbase
时,我没有看到“HMaster”进程正在运行,但是当我运行jps
时,我确实看到了HMaster。但是,我尝试启动/usr/local/opt/hbase/bin/start-hbase.sh
并输入hbase shell
,我收到了与之前相同的错误(status
)。
VI。
我已经验证我的Hadoop安装运行正常:
The node /hbase is not in ZooKeeper.
我的HBase配置似乎有问题,但我不确定还有什么可以尝试。
答案 0 :(得分:6)
由于您要设置独立的hbase
,因此无需构建hdfs
或zk
。
问题的原因在于,hbase
将尝试启动内部zk
服务,但由于已经有一个java进程QuorumPeerMain
占用端口2181
,hbase
将无法启动。如果hbase
成功启动,则会有一个名为HMaster
。
要验证这一点,请检查日志hbase-richiethomas-master-MacBook-Pro-5.local.out
的最后一行,您可能会看到如下错误消息:
Could not start ZK at requested port of 2181. ZK was started at port: 2182. Aborting as clients (e.g. shell) will not be able to find this ZK quorum.
所以,解决方案是:
QuorumPeerMain
start-hbase.sh