在本地模式下运行hadoop作业时出现错误“java.lang.IllegalArgumentException:无法从空字符串创建路径”

时间:2015-02-03 19:37:19

标签: hadoop jobs illegalargumentexception

我在Hadoop 2.2.0安装的本地模式的测试环境中定期运行hadoop作业。该作业大部分时间都成功,但偶尔会因上述错误消息而失败。调用堆栈如下:

Exception in thread "main" java.lang.IllegalArgumentException: Can not create a Path from an empty string 
    at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127) 
    at org.apache.hadoop.fs.Path.<init>(Path.java:135) 
    at org.apache.hadoop.fs.Path.<init>(Path.java:94) 
    at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:150) 
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:636) 
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:430) 
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) 
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) 
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) 
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286) 
    at com.mathworks.hadoop.MWMapReduceDriver.run(MWMapReduceDriver.java:1443) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) 
    at com.mathworks.hadoop.MWMapReduceDriver.main(MWMapReduceDriver.java:1454) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 

使用Hadoop 1.2.1,同样的工作不会失败。我查看了LocalJobRunner.java的源代码,发现第150行如下:

this.localJobDir = localFs.makeQualified(new Path(new Path(conf.getLocalPath(jobDir), user), jobid.toString()));

我在异常catch块中打印出conf.getLocalPath(jobDir),user和jobid.toString(),并在作业失败时发现用户为空。根据源代码,用户通过调用UserGroupInformation.getCurrentUser().getShortUserName()获得。有谁知道这个调用在什么条件下可以返回一个空字符串?

0 个答案:

没有答案