Spark工作成功,但是ERROR CoarseGrainedExecutorBackend:驱动程序已取消关联

时间:2016-11-17 10:49:39

标签: hadoop apache-spark yarn

我的火花版本是1.6.2并且在纱线上运行。

驱动器容器的日志报告SUCCEEDED,如下所示:

16/11/17 17:25:56 INFO ApplicationMaster: Final app status: SUCCEEDED, exitCode: 0
16/11/17 17:25:56 INFO ApplicationMaster: Unregistering ApplicationMaster with SUCCEEDED

但是,一个执行程序容器的日志记录ERROR

16/11/17 17:25:56 WARN CoarseGrainedExecutorBackend: An unknown (xxx-xxx-xxx-xxx:xxxx) driver disconnected.
16/11/17 17:25:56 ERROR CoarseGrainedExecutorBackend: Driver xxx-xxx-xxx-xxx:xxxx disassociated! Shutting down.

我认为工作成功,因为输出结果符合预期。

但我想知道为什么错误被抛出以及这项工作是否真的成功了。

我从纱线NodeManager的日志中找到更多信息:

WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception from container-launch with container ID: container_xxxxxxxxx and exit code: 1
ExitCodeException exitCode=1:
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
    at org.apache.hadoop.util.Shell.run(Shell.java:456)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

有人遇到同样的问题吗?

感谢。

0 个答案:

没有答案