Question

我正在使用带有hbase 1.0的spark 1.3.0。一周后。 Hbase使用java代码成功运行。但是当使用Hbase时会产生火花错误。我也用hbase shell检查工作正常。这个错误发生在很长一段时间后，否则也能正常工作。

我已经检查过hadoop和hbase群集健康状况良好。

at spark UI

Caused by: java.io.IOException: Enable/Disable failed 
    at org.apache.hadoop.hbase.client.ZooKeeperRegistry.isTableOnlineState(ZooKeeperRegistry.java:110) 
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isTableDisabled(ConnectionManager.java:907) 
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1076) 
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.relocateRegion(ConnectionManager.java:1064) 
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:885) 
    at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78) 
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124) 
    ... 23 more 
 Caused by:org.apache.zookeeper.KeeperException$ConnectionLossException:KeeperErrorCode = ConnectionLoss for /hbase/table/my_sample_table 
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) 
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) 
    at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151) 
    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:360) 
    at org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(ZKUtil.java:685) 
    at org.apache.hadoop.hbase.zookeeper.ZKTableStateClientSideReader.getTableState(ZKTableStateClientSideReader.java:186) 
    at org.apache.hadoop.hbase.zookeeper.ZKTableStateClientSideReader.isDisabledTable(ZKTableStateClientSideReader.java:60) 
    at org.apache.hadoop.hbase.client.ZooKeeperRegistry.isTableOnlineState(ZooKeeperRegistry.java:108) 
    ... 29 more

在某些情况下

   WARN  ZKUtil:484 - hconnection-0x792dd4040x0, quorum=Megatron:2222, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
    at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
    at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
    at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getCat org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
 clusterId(ZooKeeperRegistry.java:86)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623)
    at sun.reflect.GeneratedConstructorAccessor530.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
    at org.apache.hadoop.hbase.mapreduce.TableInputFormat.initialize(TableInputFormat.java:183)
    at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:230)
    at org.apache.hadoop.hbase.mapreduce.TableInputFormat.getSplits(TableInputFormat.java:237)
    at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:95)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
    at scala.Option.getOrElse(Option.scala:120)
    at org.apache.spark.rdd.RDD.partitions(RDD.scala:217)
    at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:32)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
    at scala.Option.getOrElse(Option.scala:120)
    at org.apache.spark.rdd.RDD.partitions(RDD.scala:217)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1511)
    at org.apache.spark.rdd.RDD.count(RDD.scala:1006)
    at  org.apache.spark.api.java.JavaRDDLike$class.count(JavaRDDLike.scala:412)
    at org.apache.spark.api.java.JavaRDD.count(JavaRDD.scala:32)

但是当尝试将某些内容放入hbase时，首先发生错误。尝试从hbase读取时出现第二个错误。

Answer 1

问题在于HBaese表锁定。处于挂起状态或处于不稳定状态。有一些建议的解决方案。

运行以下命令：

hbase hbck -fix
HMaster reboot

如果上述解决方案失败

重启大师

如果上述解决方案失败

重新启动Zookeeper

如果一切都失败

重新启动群集

可在此网址找到更多详情： JIRA Ticket

Spark无法访问hbase但能够使用java代码访问

1 个答案: