Question

我使用hbase来存储来自爬虫的数据。这些数据从hbase索引到Solr。但是我遇到了hBase的问题。由于我现在拥有更大的数据库，每次我尝试从hbase索引数据时，都会出现如下所示的错误。这是日志文件hadoop.log

的一部分

有类似的问题：这可能表示服务器连接太多（30是默认设置）。我在SO上发现这可能是maxClientConnections属性的问题所以我将此值设置为0，但此解决方案没有帮助。

2013-11-11 15:30:22,638 ERROR zookeeper.ClientCnxn - Error while calling watcher 
java.lang.NullPointerException
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.resetZooKeeperTrackers(HConnectionManager.java:315)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.abort(HConnectionManager.java:1340)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:343)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:261)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:488)
2013-11-11 15:30:27,925 WARN  mapred.FileOutputCommitter - Output path is null in cleanup
2013-11-11 15:30:27,925 WARN  mapred.LocalJobRunner - job_local1497313370_0001
org.apache.gora.util.GoraException: java.lang.RuntimeException: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to connect to ZooKeeper but the connection closes immediately. This could be a sign that the server has too many connections (30 is the default). Consider inspecting your ZK server logs for that error and then make sure you are reusing HBaseConfiguration as often as you can. See HTable's javadoc for more information.
at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:167)
at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118)
at org.apache.gora.mapreduce.GoraOutputFormat.getRecordWriter(GoraOutputFormat.java:88)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:568)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:637)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:398)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to connect to ZooKeeper but the connection closes immediately. This could be a sign that the server has too many connections (30 is the default). Consider inspecting your ZK server logs for that error and then make sure you are reusing HBaseConfiguration as often as you can. See HTable's javadoc for more information.
at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:127)
at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:102)
at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:161)
... 6 more
Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to connect to ZooKeeper but the connection closes immediately. This could be a sign that the server has too many connections (30 is the default). Consider inspecting your ZK server logs for that error and then make sure you are reusing HBaseConfiguration as often as you can. See HTable's javadoc for more information.
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1002)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:304)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:295)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:157)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90)
at org.apache.gora.hbase.store.HBaseStore.initialize(HBaseStore.java:109)
... 8 more
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:809)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:837)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:903)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:133)
... 14 more

你知道问题出在哪里吗？或者您知道我怎样才能确保我没有同时打开多个HTable实例？

Answer 1

如果有人对我如何解决这个问题感兴趣，我在hBase邮件列表中创建了一个线程并在那里解决了http://apache-hbase.679495.n3.nabble.com/hBase-the-server-has-too-many-connections-maxClientConn-property-set-to-0-does-not-help-td4052728.html#a4052873

我不知道究竟是什么解决了这个问题，但我试图将这些事情结合起来：

1）属性maxClientConnections设置为：

<property>
   <name>hbase.zookeeper.property.maxClientCnxns</name>
   <value>0</value>
</property>

2）增加时间

<property>
  <name>zookeeper.session.timeout</name>
  <value>1200000</value>
</property>

<property>
  <name>hbase.zookeeper.property.tickTime</name>
  <value>6000</value>
</property>

3）https://wiki.apache.org/nutch/ErrorMessagesInNutch2#Nutch_2.1_.2B-_HBase_0.90.4_cluster_settings_-WARN_zookeeper.ClientCnxn_-_Session_0x0_for_server_node1.xxxxxx.com.2Fxxx.xxx.xxx.xxx:2181.2C_unexpected_error.2C_closing_socket_connection_and_attempting_reconnect_java.io.IOException:_Connection_reset_by_peer

然后我杀了所有java进程，启动和停止hBase，然后运行rolling-restart.sh，现在一切正常。

Answer 2

HTable无法打开新的连接，HTable驻留在Connection对象上。如此多的HTable共享同一连接。但是您必须确保每个节点没有打开太多连接。

ZK具有一项设置，可以从一个IP地址设置最大连接数，以防止连接爆炸。默认值为30。您可以增加该值，但不能增加太多。通常，从同一主机到ZK的并发连接不应过多。否则，请使用netstat检查哪个进程打开了太多ZK连接，该进程可能存在连接泄漏。

请参阅我的其他相关答案：Error HBASE-ZOOKEEPER : Too many connections

hBase - 服务器连接太多

2 个答案: