我设置了一个双节点hadoop集群。启动群集后，它看起来像这样：
机器namenode：
hadoop@namenode:~$ jps
5691 Jps
3531 DataNode
3424 NameNode
3669 SecondaryNameNode
3822 ResourceManager
3908 NodeManager
第二台机器datanode：
hadoop@datanode:~$ jps
3716 Jps
2137 DataNode
2231 NodeManager
因此，在启动集群后，我尝试执行标准基准测试：
hadoop jar /opt/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar TestDFSIO -write -nrFiles 20 -fileSize 10
但是作业失败，配置文件包含以下消息：
在datanode：
hadoop@datanode:~$ cat /opt/hadoop-2.2.0/logs/yarn-hadoop-nodemanager-datanode.log
...
2014-02-18 16:37:41,567 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 3547 for container-id container_1392741263071_0001_02_000001: 26.2 MB of 2 GB physical memory used; 1.2 GB of 4.2 GB virtual memory used
2014-02-18 16:37:42,158 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 2 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:43,166 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 2 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:44,171 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 2 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:44,579 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 3547 for container-id container_1392741263071_0001_02_000001: 95.3 MB of 2 GB physical memory used; 1.3 GB of 4.2 GB virtual memory used
2014-02-18 16:37:45,180 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 2 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:46,183 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 2 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:47,189 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 2 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:47,584 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 3547 for container-id container_1392741263071_0001_02_000001: 108.1 MB of 2 GB physical memory used; 1.3 GB of 4.2 GB virtual memory used
2014-02-18 16:37:48,196 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 2 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:49,157 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1392741263071_0001_02_000001 is : 1
2014-02-18 16:37:49,157 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception from container-launch with container ID: container_1392741263071_0001_02_000001 and exit code: 1
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
2014-02-18 16:37:49,159 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
2014-02-18 16:37:49,159 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container exited with a non-zero exit code 1
2014-02-18 16:37:49,160 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1392741263071_0001_02_000001 transitioned from RUNNING to EXITED_WITH_FAILURE
2014-02-18 16:37:49,160 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_1392741263071_0001_02_000001
2014-02-18 16:37:49,172 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoop/hadoop/yarn-data/usercache/hadoop/appcache/application_1392741263071_0001/container_1392741263071_0001_02_000001
2014-02-18 16:37:49,173 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=hadoop OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1392741263071_0001 CONTAINERID=container_1392741263071_0001_02_000001
...
在namenode：
hadoop@namenode:/opt/hadoop-2.2.0/logs$ cat yarn-hadoop-*.log
2014-02-18 16:34:25,054 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: STARTUP_MSG:
...
2014-02-18 16:37:37,441 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 4493 for container-id container_1392741263071_0001_01_000001: 131.1 MB of 2 GB physical memory used; 1.4 GB of 4.2 GB virtual memory used
2014-02-18 16:37:38,367 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 1 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
2014-02-18 16:37:39,369 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 1 cluster_timestamp: 1392741263071 } attemptId: 1 } id: 1 } state: C_RUNNING diagnostics: "" exit_status: -1000
...
2014-02-18 16:34:23,131 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: STARTUP_MSG:
...
2014-02-18 16:37:49,186 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerNode: Released container container_1392741263071_0001_02_000001 of capacity <memory:2048, vCores:1> on host datanode.c.forward-camera-473.internal:43994, which currently has 0 containers, <memory:0, vCores:0> used and <memory:8192, vCores:8> available, release resources=true
2014-02-18 16:37:49,186 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: default used=<memory:0, vCores:0> numContainers=0 user=hadoop user-resources=<memory:0, vCores:0>
2014-02-18 16:37:49,186 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: completedContainer container=Container: [ContainerId: container_1392741263071_0001_02_000001, NodeId: datanode.c.forward-camera-473.internal:43994, NodeHttpAddress: datanode.c.forward-camera-473.internal:8042, Resource: <memory:2048, vCores:1>, Priority: 0, Token: Token { kind: ContainerToken, service: 10.240.110.76:43994 }, ] resource=<memory:2048, vCores:1> queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 usedCapacity=0.0 absoluteUsedCapacity=0.0 used=<memory:0, vCores:0> cluster=<memory:16384, vCores:16>
2014-02-18 16:37:49,186 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: completedContainer queue=root usedCapacity=0.0 absoluteUsedCapacity=0.0 used=<memory:0, vCores:0> cluster=<memory:16384, vCores:16>
2014-02-18 16:37:49,186 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting completed queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0
2014-02-18 16:37:49,186 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application appattempt_1392741263071_0001_000002 released container container_1392741263071_0001_02_000001 on node: host: datanode.c.forward-camera-473.internal:43994 #containers=0 available=8192 used=0 with event: FINISHED
2014-02-18 16:37:49,187 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: Unregistering app attempt : appattempt_1392741263071_0001_000002
2014-02-18 16:37:49,187 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1392741263071_0001_000002 State change from RUNNING to FAILED
2014-02-18 16:37:49,187 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Application application_1392741263071_0001 failed 2 times due to AM Container for appattempt_1392741263071_0001_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
.Failing this attempt.. Failing the application.
2014-02-18 16:37:49,189 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Removing info for app: application_1392741263071_0001
2014-02-18 16:37:49,194 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1392741263071_0001 State change from RUNNING to FAILED
2014-02-18 16:37:49,194 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application appattempt_1392741263071_0001_000002 is done. finalState=FAILED
2014-02-18 16:37:49,194 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo: Application application_1392741263071_0001 requests cleared
2014-02-18 16:37:49,194 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: Application removed - appId: application_1392741263071_0001 user: hadoop queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0
2014-02-18 16:37:49,194 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Application removed - appId: application_1392741263071_0001 user: hadoop leaf-queue of parent: root #applications: 0
2014-02-18 16:37:49,204 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hadoop OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1392741263071_0001 failed 2 times due to AM Container for appattempt_1392741263071_0001_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
.Failing this attempt.. Failing the application. APPID=application_1392741263071_0001
2014-02-18 16:37:49,205 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: appId=application_1392741263071_0001,name=hadoop-mapreduce-client-jobclient-2.2.0-tests.jar,user=hadoop,queue=default,state=FAILED,trackingUrl=namenode:8088/cluster/app/application_1392741263071_0001,appMasterHost=,startTime=1392741381131,finishTime=1392741469188,finalStatus=FAILED
2014-02-18 16:37:49,205 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Cleaning master appattempt_1392741263071_0001_000002
发生了什么事？
答案 0 :(得分：0)
看起来它不能产生新的java进程。可能你的.profile或.bashrc没有正确设置JAVA_HOME或PATH，因此无法访问java可执行文件。