Question

我正在尝试使用Hadoop，YARN和Accumulo运行MapReduce作业。

我得到以下输出，我无法追查问题。看起来是一个YARN问题，但我不确定它在寻找什么。我在$ HADOOP_PREFIX / grid / hadoop / hdfs / yarn / logs位置有一个nmPrivate文件夹。这是它说无法找到的文件夹吗？

14/03/31 08:48:46 INFO mapreduce.Job: Job job_1395942264921_0023 failed with state FAILED due to: Application application_1395942264921_0023 failed 2 times due to AM Container for appattempt_1395
942264921_0023_000002 exited with  exitCode: -1000 due to: Could not find any valid local directory for nmPrivate/container_1395942264921_0023_02_000001.tokens
.Failing this attempt.. Failing the application.

Answer 1

当我在集群模式下测试“ spark-submit-on-yarn”时：

spark-submit --master yarn --deploy-mode cluster --class org.apache.spark.examples.SparkPi /usr/local/install/spark-2.2.0-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.2.0.jar 100

我遇到了同样的错误：

Application application_1532249549503_0007 failed 2 times due to AM Container for appattempt_1532249549503_0007_000002 exited with exitCode: -1000 Failing this attempt.Diagnostics: java.io.IOException: Resource file:/usr/local/install/spark-2.2.0-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.2.0.jar changed on src filesystem (expected 1531576498000, was 1531576511000

存在一个建议来解决这种错误，以修改您的core-site.xml或HADOOP的其他conf。

最后，我通过在$ HADOOP_HOME / etc / hadoop / core-site.xml中设置属性fs.defaultFS来解决错误

Hadoop / YARN作业失败 - “退出时使用exitCode：-1000，原因是：找不到nmPrivate的任何有效本地目录...”

1 个答案: