Question

我一直在尝试使用CDH中的Heapspace对此问题进行分类。我查看了Stackoverflow论坛，但没有任何工作。我已经启动了Hive服务器，我正在尝试登录Hive并运行一个基本命令。以下是问题的快照。

$ sudo service hive-server start

Starting
(hive-server):                                    [  OK  ]
hive Logging initialized using
configuration in file:/etc/hive/conf.dist/hive-log4j.properties 
Hive history file=/tmp/dmxdemo/hive_job_log_dmxdemo_201404221719_901641535.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J:
See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 

hive> show tables; 
java.lang.OutOfMemoryError: Java heap space 
  at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:353)
  at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_database(ThriftHiveMetastore.java:412)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:399)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:736)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)    
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)   
  at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74)
  at $Proxy9.getDatabase(Unknown Source)    
  at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1114)
  at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1103)
  at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:2206)
  at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:334)
  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)     
  at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1352)  
  at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1138)     
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951)  
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)   
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)  
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)    
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)   
  at org.apache.hadoop.util.RunJar.main(RunJar.java:208) FAILED: Execution
Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask

我在hadoop-env.sh中定义的环境变量是：

# The maximum amount of heap to use, in MB. Default is 1000.
export HADOOP_HEAPSIZE=3096
#export HADOOP_NAMENODE_INIT_HEAPSIZE=""

# Extra Java runtime options.  Empty by default.
#export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true $HADOOP_CLIENT_OPTS"
export HADOOP_CLIENT_OPTS="-Xmx2048m $HADOOP_CLIENT_OPTS"

hive-env.sh的定义：

if [ "$SERVICE" = "cli" ]; then
   if [ -z "$DEBUG" ]; then
     export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRat
io=40 -XX:MinHeapFreeRatio=15 -XX:+UseParNewGC -XX:-UseGCOverheadLimit"
   else
     export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRat
io=40 -XX:MinHeapFreeRatio=15 -XX:-UseGCOverheadLimit"
   fi
fi

# The heap size of the jvm stared by hive shell script can be controlled via:
#
 export HADOOP_HEAPSIZE=3096
#

问题出在哪里？请指教。

Hive Issue - java.lang.OutOfMemoryError：Java堆空间

0 个答案: