我的hadoop版本是2.5.0
按照指南http://www.elastic.co/guide/en/elasticsearch/hadoop/2.1.Beta/ey-usage.html
我下载了elasticsearh-hadoop-2.1.0beta4.zip。
我在下面运行hadoop jar elasticsearh-yarn-2.1.0beta4.jar。
hadoop jar elasticsearh-yarn-2.1.0beta4.jar -download-es
hadoop jar elasticsearh-yarn-2.1.0beta4.jar -install-es
hadoop jar elasticsearh-yarn-2.1.0beta4.jar -install
已下载弹性1.5.2。 弹性的1.5.5.zip和elasticsearh-yarn-2.1.0beta4.jar已上传到hdfs:// host:port / apps / elasticsearch。
当我用纱线开始es时,输出很好。
hadoop jar /tmp/elasticsearch-hadoop-2.1.0.Beta4/dist/elasticsearch-yarn-2.1.0.Beta4.jar -start
15/05/13 16:50:09 INFO client.RMProxy: Connecting to ResourceManager at hdp1/10.80.3.231:8032
15/05/13 16:50:11 INFO impl.YarnClientImpl: Submitted application application_1431506015778_0003
Launched a 1 node Elasticsearch-YARN cluster [application_1431506015778_0003@http://xxxx:8088/proxy/application_1431506015778_0003/] at ......
但是当我稍后检查状态时,应用程序失败了。
hadoop jar /tmp/logsystem/elasticsearch-hadoop-2.1.0.Beta4/dist/elasticsearch-yarn-2.1.0.Beta4.jar -status
INFO client.RMProxy: Connecting to ResourceManager at hdp1/10.80.3.231:8032
Id State Status Start Time Finish Time Tracking URL
application_1431506015778_0001 FINISHED FAILED 15-5-13 PM 4:36 15-5-13 下午4:37 http://xxx:8088/proxy/application_1431506015778_0001/A
application_1431506015778_0002 FINISHED FAILED 15-5-13 PM 4:49 15-5-13 下午4:49 http://xxxx:8088/proxy/application_1431506015778_0002/A
我用纱线检查了日志,AM的日志正在打击
15/05/13 16:50:18 INFO am.ApplicationMaster: Starting ApplicationMaster...
15/05/13 16:50:19 INFO client.RMProxy: Connecting to ResourceManager at hdp1/10.80.3.231:8030
15/05/13 16:50:19 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
15/05/13 16:50:19 INFO am.EsCluster: Allocating Elasticsearch cluster with 1 nodes
15/05/13 16:50:24 INFO impl.AMRMClientImpl: Received new token for : hdp4:8041
15/05/13 16:50:26 INFO am.EsCluster: About to launch container for command: [{{SHELL}} elasticsearch-1.5.2.zip/elasticsearch-1.5.2/bin/elasticsearch 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr]
15/05/13 16:50:26 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hdp4:8041
15/05/13 16:50:26 INFO am.EsCluster: Started container Container: [ContainerId: container_1431506015778_0003_01_000002, NodeId: xxxx:8041, NodeHttpAddress: hdp4:8042, Resource: <memory:2048, vCores:1>, Priority: -1, Token: Token { kind: ContainerToken, service: XXXX:8041 }, ]
15/05/13 16:50:26 INFO am.EsCluster: Fully allocated 1 containers
**15/05/13 16:50:31 WARN am.EsCluster: Container container_1431506015778_0003_01_000002 exited with an invalid/unknown exit code...**
15/05/13 16:50:31 WARN am.EsCluster: Cluster has not completed succesfully...
15/05/13 16:50:31 INFO am.EsCluster: Cluster has completed running...
15/05/13 16:50:46 INFO impl.ContainerManagementProtocolProxy: Opening proxy : hdp4:8041
15/05/13 16:50:46 INFO impl.AMRMClientImpl: Waiting for application to be successfully unregistered.
在nodemanager的日志中,唯一的错误信息是exitcode 3
2015-05-13 16:50:26,083 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1431506015778_0003_01_000002 is : 3
2015-05-13 16:50:26,084 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception from container-launch with container ID: container_1431506015778_0003_01_000002 and exit code: 3
ExitCodeException exitCode=3:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
2015-05-13 16:50:26,085 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception from container-launch.
我的问题是:
1,es-hadoop 2.1.0beta4和es 1.5.2能否运行hadoop-2.5.0?
2,我是否需要使用hadoop-2.5.0重建jar?
3,exitcode 3是什么意思?
4,如何获取有关Container中错误的更多信息?
答案 0 :(得分:0)
尝试查看应用程序日志,这些日志通常包含更多信息:
yarn logs -applicationId application_1431506015778_0002