执行sqoop作业时找不到文件错误

时间:2018-05-29 12:37:54

标签: hadoop import sqoop

当我执行sqoop作业时,它会抛出一个FileNotFoundException错误,如下所示

  

18/05/29 06:18:59 INFO orm.CompilationManager:编写jar文件:/tmp/sqoop-hduser/compile/0ce66d1f09ce960a71c165855afbe42c/QueryResult.jar   18/05/29 06:18:59 INFO mapreduce.ImportJobBase:开始查询导入。   18/05/29 06:18:59 INFO Configuration.deprecation:不推荐使用mapred.job.tracker。相反,请使用mapreduce.jobtracker.address   18/05/29 06:18:59 WARN util.NativeCodeLoader:无法为您的平台加载native-hadoop库...使用适用的builtin-java类   18/05/29 06:18:59 INFO Configuration.deprecation:不推荐使用mapred.jar。相反,请使用mapreduce.job.jar   18/05/29 06:19:01 INFO Configuration.deprecation:不推荐使用mapred.job.tracker。相反,请使用mapreduce.jobtracker.address   18/05/29 06:19:01 INFO Configuration.deprecation:不推荐使用mapred.map.tasks。相反,请使用mapreduce.job.maps   18/05/29 06:19:01 INFO Configuration.deprecation:不推荐使用session.id.相反,请使用dfs.metrics.session-id   18/05/29 06:19:01 INFO jvm.JvmMetrics:使用processName = JobTracker初始化JVM指标,sessionId =   18/05/29 06:19:01 INFO mapreduce.JobSubmitter:清理临时区域文件:/app/hadoop/tmp/mapred/staging/hduser1354549662/.staging/job_local1354549662_0001   18/05/29 06:19:01 ERROR tool.ImportTool:遇到IOException运行导入作业:java.io.FileNotFoundException:文件不存在:hdfs:// svn-server:54310 / home / hduser / sqoop-1.4。 6.bin__hadoop-2.0.4-α/ LIB / PostgreSQL的-9.2-1002-jdbc4.jar       在org.apache.hadoop.hdfs.DistributedFileSystem $ 18.doCall(DistributedFileSystem.java:1122)       在org.apache.hadoop.hdfs.DistributedFileSystem $ 18.doCall(DistributedFileSystem.java:1114)       在org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)       在org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)       at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)       at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)       at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:99)       at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)       at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:269)       at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:390)       在org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:483)       在org.apache.hadoop.mapreduce.Job $ 10.run(Job.java:1296)       在org.apache.hadoop.mapreduce.Job $ 10.run(Job.java:1293)       at java.security.AccessController.doPrivileged(Native Method)       在javax.security.auth.Subject.doAs(Subject.java:415)       在org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)       在org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)       在org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)       在org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196)       在org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169)       在org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)       at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:729)       在org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)       在org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)       在org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)       在org.apache.sqoop.tool.JobTool.run(JobTool.java:283)       在org.apache.sqoop.Sqoop.run(Sqoop.java:143)       在org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)       在org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)       在org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)       在org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)       在org.apache.sqoop.Sqoop.main(Sqoop.java:236)   `

它应该在本地sqoop/lib目录中查找jar和其他依赖项,但它在HDFS中查找它与我的本地sqoop lib路径具有相同的文件路径。根据项目要求,我需要sqoop来查看我的本地库。我怎样才能做到这一点?谢谢。

1 个答案:

答案 0 :(得分:0)

您是否在.bashrc文件中为sqoop设置了 SQOOP_HOME PATH 环境变量

如果没有请添加

  

vi~ / .bashrc

包括以下行

export SQOOP_HOME=/your/path/to/the/sqoop

export PATH=$PATH:$SQOOP_HOME/bin

保存文件并执行以下命令

  

source~ / .bashrc

希望这对你有所帮助!!!