由于java.io.FileNotFoundException而导致MapReduce应用程序失败

时间:2018-08-23 14:14:50

标签: java hadoop filenotfoundexception

我尝试在Hadoop集群上运行我的应用程序,并收到以下错误:

    java.io.FileNotFoundException: File file:/dfs/1/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 does not exist
        at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:624)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:850)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:614)
        at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1073)
        at org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:176)
        at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:198)
        at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:732)
        at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:728)
        at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
        at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:735)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createDir(ContainerLocalizer.java:428)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:420)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:134)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.main(ContainerLocalizer.java:389)

我的日志中也充满了这样的消息:

    Application application_1532002950099_733363 initialization failed (exitCode=255) with output: main : command provided 0
    main : run as user is mvzakharchenko
    main : requested yarn user is mvzakharchenko
    Can't create directory /dfs/1/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/11/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/12/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/2/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/3/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/4/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/5/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/6/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/7/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/8/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied
    Can't create directory /dfs/9/hadoop/yarn/local/usercache/mvzakharchenko/appcache/application_1532002950099_733363 - Permission denied

我该如何解决?

0 个答案:

没有答案