一段时间后,Hbase正在冻结

时间:2015-08-04 13:45:00

标签: hbase

我们的hbase应用程序运行了几年,现在每天冻结3-4次,我们必须重新启动tomcat才能使我们的服务恢复正常。你能告诉我这次失败的原因是什么吗?以下是我们得到的堆栈跟踪: -

[2015-07-29 16:11:53,850] WARN246124[pool-1-thread-8] - org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1599) - Failed all from region=Session_EG36499597,A95e3582dd-985b-4bbf-8c9c-c3e7378b2c9c~2014111717~B41ea71319-6c87-414e-83e9-3260da33dbce,1429545209762.33fb72a8c67d79dfe2743542b29511a5., hostname=csotg11-dn-rs-tt, port=60020
java.util.concurrent.ExecutionException: java.io.IOException: Call to csotg11-dn-rs-tt/192.168.1.56:60020 failed on local exception: java.nio.channels.ClosedByInterruptException
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1569)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1421)
        at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:955)
        at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:811)
        at org.apache.hadoop.hbase.client.HTable.put(HTable.java:786)
        at org.apache.hadoop.hbase.client.HTablePool$PooledHTable.put(HTablePool.java:394)
        at egain.DAO.UserDAO.SaveUser(UserDAO.java:156)
        at egain.services.TrackerTask.call(TrackerTask.java:38)
        at egain.asyncManagement.UpdateUserRequest.run(UpdateUserRequest.java:22)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
Caused by: java.io.IOException: Call to csotg11-dn-rs-tt/192.168.1.56:60020 failed on local exception: java.nio.channels.ClosedByInterruptException
        at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:1030)
        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:999)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
        at com.sun.proxy.$Proxy60.multi(Unknown Source)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1398)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1396)
        at org.apache.hadoop.hbase.client.ServerCallable.withoutRetries(ServerCallable.java:210)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1405)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1393)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        ... 3 more
Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:473)
        at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:62)
        at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:143)
        at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:153)
        at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:114)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at java.io.DataOutputStream.flush(DataOutputStream.java:123)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:620)
        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
        at com.sun.proxy.$Proxy60.openScanner(Unknown Source)
        at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:218)
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:120)
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
        at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:163)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:205)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:120)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:665)
        at org.apache.hadoop.hbase.client.HTablePool$PooledHTable.getScanner(HTablePool.java:378)
        at egain.models.HBaseConnectionFactory.getScanner(HBaseConnectionFactory.java:313)
        at egain.DAO.SessionDAO.getSession(SessionDAO.java:44)
        at egain.DAO.SessionDAO.getCompleteDataForSession(SessionDAO.java:109)
        at egain.utils.eGainParser.getBySessionId(eGainParser.java:54)
        at egain.services.DataTask.call(DataTask.java:52)
        at egain.services.DataTask.call(DataTask.java:1)
        ... 5 more

0 个答案:

没有答案