Question

当我运行hadoop作业时，它失败并带有以下stacktrace：

11/10/06 13:12:49 INFO mapred.FileInputFormat: Total input paths to process : 1
11/10/06 13:12:49 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/Har/.staging/job_201110051450_0007
11/10/06 13:12:49 ERROR streaming.StreamJob: Error Launching job : java.io.IOException: java.io.IOException: No valid local directories in property: mapred.local.dir
    at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3923)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:523)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1383)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1379)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1377)
Caused by: java.io.IOException: No valid local directories in property: mapred.local.dir
    at org.apache.hadoop.conf.Configuration.getLocalPath(Configuration.java:948)
    at org.apache.hadoop.mapred.JobConf.getLocalPath(JobConf.java:457)
    at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:401)
    at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3921)
    ... 11 more

Streaming Job Failed!

我该如何解决这个问题？

谢谢

Answer 1

mapred.local.dir需要正确配置。检查是否在mapred-site或hdfs-site中配置。该目录应该是现有的。执行作业的用户应具有此目录的适当权限

Hadoop：java.io.IOException：属性中没有有效的本地目录：mapred.local.dir

1 个答案: