Question

我有一个32位的Linux系统在3gb ram上运行。当我试图运行hadoop示例时，它没有说没有足够的内存来分配jre。生成的结果是：

hadoop jar mapreduce/hadoop-mapreduce-examples-*.jar grep input output ‘dfs[a-z.]+’
15/01/11 10:17:04 INFO client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8032
15/01/11 10:17:05 WARN mapreduce.JobSubmitter: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
15/01/11 10:17:05 INFO input.FileInputFormat: Total input paths to process : 7
15/01/11 10:17:06 INFO mapreduce.JobSubmitter: number of splits:7
15/01/11 10:17:06 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name

15/01/11 10:17:06 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
15/01/11 10:17:06 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
15/01/11 10:17:06 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
15/01/11 10:17:06 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
15/01/11 10:17:06 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
15/01/11 10:17:06 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
15/01/11 10:17:06 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
15/01/11 10:17:06 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
15/01/11 10:17:06 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
15/01/11 10:17:06 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
15/01/11 10:17:06 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
15/01/11 10:17:07 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1420951126090_0001
15/01/11 10:17:07 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
15/01/11 10:17:07 INFO impl.YarnClientImpl: Submitted application application_1420951126090_0001 to ResourceManager at /127.0.0.1:8032
15/01/11 10:17:07 INFO mapreduce.Job: The url to track the job: http://:8088/proxy/application_1420951126090_0001/
15/01/11 10:17:07 INFO mapreduce.Job: Running job: job_1420951126090_0001
15/01/11 10:17:16 INFO mapreduce.Job: Job job_1420951126090_0001 running in uber mode : false
15/01/11 10:17:16 INFO mapreduce.Job:  map 0% reduce 0%
Java HotSpot(TM) Server VM warning: INFO: os::commit_memory(0xa7b5d000, 32768, 1) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 32768 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/local/hadoop-2.2.0/share/hadoop/hs_err_pid5496.log
#
# Compiler replay data is saved as:
# /usr/local/hadoop-2.2.0/share/hadoop/replay_pid5496.log

是因为我的配置硬件还是hadoop中的一些设置错误？

Answer 1

这表示您已用完虚拟内存，尝试增加交换空间，或减少堆以使程序的其余部分保留虚拟内存。一个32位的程序被限制为大约3 GB的虚拟内存总量，所以如果它全部分配给堆，这对程序运行起来不会留下太多。相比之下，64位程序仅限于128 TB到256 TB，具体取决于操作系统。

BTW在Windows上，32位程序仅限于大约1.5 GB的虚拟内存。

由于hadoop是一种大数据解决方案，它通常在更大的机器上运行。例如256 GB到1 TB并不罕见。鉴于目前32 GB非常便宜，我会考虑至少获得这么多，或者更多的内存。

运行Hadoop：Java Runtime Environment的内存不足以继续

1 个答案: