我们正在运行map reduce / spark作业,以便在其中一个环境中批量加载hbase数据

时间:2016-05-10 15:00:56

标签: apache-spark hbase

我们正在运行map reduce/spark作业,以便在其中一个环境中批量加载hbase数据。

在运行它时,与hbase zookeeper的连接无法初始化抛出以下错误。

    16/05/10 06:36:10 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=c321shu.int.westgroup.com:2181,c149jub.int.westgroup.com:2181,c167rvm.int.westgroup.com:2181 sessionTimeout=90000 watcher=hconnection-0x74b47a30, quorum=c321shu.int.westgroup.com:2181,c149jub.int.westgroup.com:2181,c167rvm.int.westgroup.com:2181, baseZNode=/hbase
    16/05/10 06:36:10 INFO zookeeper.ClientCnxn: Opening socket connection to server c321shu.int.westgroup.com/10.204.152.28:2181. Will not attempt to authenticate using SASL (unknown error)
    16/05/10 06:36:10 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /10.204.24.16:35740, server: c321shu.int.westgroup.com/10.204.152.28:2181
    16/05/10 06:36:10 INFO zookeeper.ClientCnxn: Session establishment complete on server c321shu.int.westgroup.com/10.204.152.28:2181, sessionid = 0x5534bebb441bd3f, negotiated timeout = 60000
    16/05/10 06:36:11 INFO mapreduce.HFileOutputFormat2: Looking up current regions for table ecpdevv1patents:NormNovusDemo
    Exception in thread "main" org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions:
    Tue May 10 06:36:11 CDT 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@3927df20, java.io.IOException: Call to c873gpv.int.westgroup.com/10.204.67.9:60020 failed on local exception: java.io.EOFException

我们在Titan DEV也执行了相同的工作,但面临同样的问题。如果有人以前遇到过同样的问题,请告诉我们。

详情是,

•之前的工作未能连接到localhost/127.0.0.1:2181。因此,我们从hbase.zookeeper.quorum获得的c149jub.int.westgroup.com,c321shu.int.westgroup.com,c167rvm.int.westgroup.com地图缩减代码中仅设置了属性hbase-site.xml。 •我们使用的是cdh version 5.3.3

0 个答案:

没有答案