Question

我对Hbase很新。我在独立模式下使用Hbase并使用谷歌n-gram开发应用程序。我在Amazon Ec2实例中使用了一个ubuntu框

我正在通过批处理API使用我的程序进行1M次写入。

        HTable table = new HTable(conf, tableName);
        conf.set("zookeeper.session.timeout", "180000");
        conf.set("hbase.rpc.timeout", "180000");
        conf.set("hbase.regionserver.logroll.period", "180000");
        conf.set("hbase.hregion.max.filesize","51200");
        table.setAutoFlush(false);
        table.batch(iterateList);
        table.flushCommits();

当我用1M行执行此操作时，我将按以下顺序获得以下异常，

    12/08/28 14:17:13 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x1396d9653bd0004, likely server has closed socket, closing socket connection and attempting reconnect
    12/08/28 14:17:13 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x1396d9653bd0003, likely server has closed socket, closing socket connection and attempting reconnect
    12/08/28 14:17:13 WARN client.HConnectionManager$HConnectionImplementation: Failed all from region=googlegrams,,1346160551634.abc8baf335dee8485d64a6c8b989d6b0., hostname=ip-10-80-74-132.ec2.internal, port=45549

java.util.concurrent.ExecutionException：java.io.IOException：在本地异常上调用ip-10-80-74-132.ec2.internal / 10.80.74.132：45549失败：java.io.EOFException

然后

  12/08/28 14:17:15 WARN zookeeper.ClientCnxn: Session 0x1396d9653bd0004
  for server null, unexpected error, closing socket connection and attempting reconnect
  java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)

我试过的很少的东西， 1. ubuntu的ulimit和nproc设置的更改 2. Hbase配置，如zookeeper超时，rpc超时日志周期和hregion fileSize

请帮忙！

干杯， Dwarak

Answer 1

尝试增加hbase.client.lease.period

Hbase批量上传问题 - 独立模式

1 个答案: