Question

我创建了扩展配置单元表：

 CREATE EXTERNAL TABLE  hivelog (timestamp BIGINT ,value STRING)
 PARTITIONED BY(dt BIGINT) row format serde
 'hive.serde.AdsClickLogWritableDeserializer'  stored as inputformat
 'org.apache.hadoop.mapred.SequenceFileInputFormat'   outputformat
 'org.apache.hadoop.mapred.SequenceFileOutputFormat' location
 '/user/hive/HiveData';

hive.serde.AdsClickLogWritableDeserializer is my class deserialized data.

然后我运行查询

select count(*) from hivelog  that has error:

2013-04-22 22:30:51,228 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.OutOfMemoryError: Java heap space
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

我的hive版本是0.90，hadoop版本是1.0.0

如何解决这个问题。非常感谢。

Answer 1

尝试增加JVM内存设置以处理OutOfMemoryError

参考[如何增加JVM内存？] [1]和[内存不足时JVM的行为错误？] [2] [1]：How can I increase the JVM memory? [2]：Behaviour of JVM during out of memory error? List s = new ArrayList<String>();

配置单元错误运行选择查询

1 个答案: