Question

我正在使用spark streaming job来执行多个任务。它运行正常大约5-6小时，但之后它失败了以下异常。 Spark流式传输作业在具有20 GB RAM和8核心的纱线集群上运行。

Application application_1435667829099_0003 failed 2 times due to AM Container for appattempt_1435667829099_0003_000002 exited with exitCode: 11
For more detailed output, check application tracking page:http://hdp-master:8088/proxy/application_1435667829099_0003/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e09_1435667829099_0003_02_000001
Exit code: 11
Stack trace: ExitCodeException exitCode=11:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 11
Failing this attempt. Failing the application.

请建议我。

Answer 1

在我的情况下，在node-manager的日志中有异常之前的行：

seq<Runner>

但我确信这是原始问题的缺点。仔细查看节点管理器日志

Answer 2

我不确定我找到的解决方案是否可以解决您的问题。但是，它解决了与exitCode: 11 (reason: Max number of executor failures (16) reached)

使用代码11退出Spark流工作

2 个答案: