Question

在提交地图时收到以下消息。我用-XX：MaxPermSize = 128m内存大小开始了我的map reduce程序。

有没有人知道现在有什么线索 -

17/03/24 09:58:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 1160328 for svc_pffr on ha-hdfs:nameservice3
    17/03/24 09:58:46 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
    17/03/24 09:58:46 INFO security.TokenCache: Got dt for hdfs://nameservice3; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nameservice3, Ident: (HDFS_DELEGATION_TOKEN token 1160328 for svc_pffr)
    17/03/24 09:58:46 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
    17/03/24 09:58:46 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
    17/03/24 09:58:47 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
    17/03/24 10:01:55 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/svc_pffr/.staging/job_1489708003568_5870
    Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
            at org.apache.hadoop.security.token.Token.<init>(Token.java:85)
            at org.apache.hadoop.hdfs.protocol.LocatedBlock.<init>(LocatedBlock.java:52)
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.convert(PBHelper.java:755)
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.convertLocatedBlock(PBHelper.java:1174)
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.convert(PBHelper.java:1192)
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.convert(PBHelper.java:1328)
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.convert(PBHelper.java:1436)
            at org.apache.hadoop.hdfs.protocolPB.PBHelper.convert(PBHelper.java:1445)
            at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:549)
            at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
            at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
            at com.sun.proxy.$Proxy23.getListing(Unknown Source)
            at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1893)
            at org.apache.hadoop.hdfs.DistributedFileSystem$15.<init>(DistributedFileSystem.java:742)
            at org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:731)
            at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1664)
            at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:300)
            at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:264)
            at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)
            at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:385)
            at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:589)
            at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:606)
            at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:490)
            at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
            at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:415)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
            at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)

Answer 1

您需要增加客户端堆大小。

你能尝试： - 导出HADOOP_CLIENT_OPTS =“$ HADOOP_CLIENT_OPTS -Xmx1g”。

然后在没有 XX：MaxPermSize = 128m

的情况下开始执行hadoop

您也可以查看此链接.. http://community.cloudera.com/t5/Batch-Processing-and-Workflow/cloudera-hadoop-mapreduce-job-GC-overhead-limit-exceeded-error/td-p/14754

Answer 2

我有这个例外，我只是格式化了我的hdfs因为它已经饱和了！

$ hadoop namenode -format

请注意：如果格式化hdfs，您将丢失与数据节点相关的所有元数据，因此数据节点上的所有信息都将丢失！

Answer 3

实际上在我的情况下 - 我得到6000个输入路径，我的map-reduce程序将创建最少6000个映射器，因此我在提交时出现内存异常。有没有什么好方法可以解决这类问题。

Answer 4

我知道这听起来很奇怪，但是我遇到了同样的错误，并且找到了原因。对我来说，是firehose_non_java_memory_bytes内部的Cloudera Management Service设置得太高了。

我不知道这是否可以解决您的问题，可能值得尝试。

获取java.lang.OutOfMemoryError：超出GC开销限制提交Map Reduce时

4 个答案: