FAILED:执行错误,从org.apache.hadoop.hive.ql.exec.mr.MapRedTask返回代码1。没有这样的文件或目录

时间:2017-07-18 02:58:04

标签: hive

在Hive上运行简单SELECT COUNT(*) FROM Table

时获取以下异常
  

作业提交因异常而失败   ' org.apache.hadoop.io.nativeio.NativeIOException(没有这样的文件或   目录)' FAILED:执行错误,返回代码1   org.apache.hadoop.hive.ql.exec.mr.MapRedTask。没有这样的文件或   目录

简单SELECT * FROM Table不会出现问题。  
请说明可能出现的问题。 Hive执行引擎是MR。

完整堆栈错误

  

2017-07-18T07:18:52,744 ERROR [main]:exec.Task(:()) - 工作提交   失败,例外   ' org.apache.hadoop.io.nativeio.NativeIOException(没有这样的文件或   目录)' ENOENT:没有这样的文件或目录   org.apache.hadoop.io.nativeio.NativeIO $ POSIX.chmodImpl(原生方法)     在   org.apache.hadoop.io.nativeio.NativeIO $ POSIX.chmod(NativeIO.java:230)     在   org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:729)     在   org.apache.hadoop.fs.ChecksumFileSystem $ 1.适用(ChecksumFileSystem.java:505)     在   org.apache.hadoop.fs.ChecksumFileSystem $ FsOperation.run(ChecksumFileSystem.java:486)     在   org.apache.hadoop.fs.ChecksumFileSystem.setPermission(ChecksumFileSystem.java:502)     在org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:602)at   org.apache.hadoop.mapreduce.JobResourceUploader.uploadFiles(JobResourceUploader.java:94)     在   org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:95)     在   org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:190)     在org.apache.hadoop.mapreduce.Job $ 10.run(Job.java:1290)at at   org.apache.hadoop.mapreduce.Job $ 10.run(Job.java:1287)at at   java.security.AccessController.doPrivileged(Native Method)at   javax.security.auth.Subject.doAs(Subject.java:415)at   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)     在org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)at   org.apache.hadoop.mapred.JobClient $ 1.run(JobClient.java:575)at at   org.apache.hadoop.mapred.JobClient $ 1.run(JobClient.java:570)at at   java.security.AccessController.doPrivileged(Native Method)at   javax.security.auth.Subject.doAs(Subject.java:415)at   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)     在   org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)     在org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)     在   org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:433)     在   org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:138)     在org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)at   org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)     在org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)at   org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)at   org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)at   org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)at at   org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)at at   org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)     在   org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)     在   org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)     在   org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)     在org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)at   org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)at at   sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at   sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)     在   sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     在java.lang.reflect.Method.invoke(Method.java:606)at   org.apache.hadoop.util.RunJar.run(RunJar.java:221)at   org.apache.hadoop.util.RunJar.main(RunJar.java:136)

     

2017-07-18T07:18:52,745 ERROR [main]:ql.Driver(:()) - 失败:   执行错误,返回代码1   org.apache.hadoop.hive.ql.exec.mr.MapRedTask。没有这样的文件或   目录

1 个答案:

答案 0 :(得分:2)

尝试检查hadoop tmp dir的权限。例如,在core-site.xml中配置了此目录的路径。属性名称为hadoop.tmp.dir。 我遇到了与您描述的问题类似的问题,它是由用户引起的,在该用户下执行MR作业,没有对该位置的写入权限。