Question

我在一台独立的机器上运行配置单元。 Hadoop以伪分布式模式运行。我正在运行hive查询，它连接两个表（一个表有7M，另一个表有51M记录，每个包含8列）。处理一段时间后，Mapper达到零百分比，然后偶尔继续打印零。你能帮我解决这个问题。

供参考，查找以下日志。

2016-04-12 22:52:58,469 Stage-1 map = 71%,  reduce = 1%
2016-04-12 22:53:00,517 Stage-1 map = 72%,  reduce = 1%
2016-04-12 22:53:02,560 Stage-1 map = 73%,  reduce = 1%
2016-04-12 22:53:09,740 Stage-1 map = 74%,  reduce = 1%
2016-04-12 22:53:11,796 Stage-1 map = 75%,  reduce = 1%
2016-04-12 22:53:13,842 Stage-1 map = 76%,  reduce = 1%
2016-04-12 22:53:21,037 Stage-1 map = 77%,  reduce = 1%
2016-04-12 22:53:24,114 Stage-1 map = 78%,  reduce = 1%
2016-04-12 22:53:26,156 Stage-1 map = 79%,  reduce = 1%
2016-04-12 22:53:35,433 Stage-1 map = 81%,  reduce = 1%
2016-04-12 22:53:38,507 Stage-1 map = 82%,  reduce = 1%
2016-04-12 22:53:45,725 Stage-1 map = 82%,  reduce = 0%
2016-04-12 22:53:49,925 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:54:50,236 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:55:50,546 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:56:50,863 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:57:51,128 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:58:51,352 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:59:51,612 Stage-1 map = 0%,  reduce = 0%
2016-04-12 23:00:51,886 Stage-1 map = 0%,  reduce = 0%
2016-04-12 23:01:52,131 Stage-1 map = 0%,  reduce = 0%

我在跟踪器中验证了状态。状态显示两次尝试，一次尝试失败，诊断消息如下。

AM Container for appattempt_1460481465127_0001_000001 exited with exitCode: -100
For more detailed output, check application tracking page:http://localhost:8088/cluster/app/application_1460481465127_0001Then, click on links to logs of each attempt.
Diagnostics: Container released on a *lost* nodeFailing this attempt

提前致谢。

Answer 1

问题似乎是由地图方面的堆空间引起的。
尝试通过执行以下操作来增加映射任务堆大小：

mapred-site.xml中的

（尝试调整以下值以与您的用例匹配）：

<!-- for mappers -->
<property>
   <name>mapreduce.map.memory.mb</name>
   <value>4096</value>
</property>
<!-- For reduces -->
<property>
   <name>mapreduce.reduce.memory.mb</name>
   <value>8192</value>
</property>

Hive Job处理在某个时间处理后停止

1 个答案: