我已经使用Nutch和2.3.1配置了Apache Hadoop 2.7.4，Hbase 1.2.6和Solr 6.6.1。我所有节点上的mapred-site.xml都是
<property>
<name>mapreduce.jobtracker.address</name>
<value>master:9001</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.task.timeout</name>
<value>10000</value>
<description>The number of milliseconds before a task will be
terminated if it neither reads an input, writes an output, nor
updates its status string. A value of 0 disables the timeout.
</description>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>2048</value>
<description>The amount of memory to request from the scheduler for each
reduce task.
</description>
</property>
和yarn-site.xml在所有节点上都有以下配置。
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>3072</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>2</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>1024</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-vcores</name>
<value>1</value>
</property>
所有作业都能成功运行，但在solrindex时，它失败并出现以下异常
17/10/13 12:30:43 INFO mapreduce.Job: map 0% reduce 0%
17/10/13 12:35:12 INFO mapreduce.Job: Task Id : attempt_1507879413631_0001_m_000000_1, Status : FAILED
AttemptID:attempt_1507879413631_0001_m_000000_1 Timed out after 10 secs
17/10/13 12:39:42 INFO mapreduce.Job: Task Id : attempt_1507879413631_0001_m_000000_2, Status : FAILED
AttemptID:attempt_1507879413631_0001_m_000000_2 Timed out after 10 secs
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
17/10/13 12:44:13 INFO mapreduce.Job: map 100% reduce 0%
17/10/13 12:44:13 INFO mapreduce.Job: Job job_1507879413631_0001 failed with state FAILED due to: Task failed task_1507879413631_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0
奴隶纱线日志之一有以下信息
2017-10-13 12:30:12,442 INFO [IPC Server handler 0 on 37429] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1507879413631_0001_m_000000_0 is : 0.0
2017-10-13 12:30:15,450 INFO [IPC Server handler 5 on 37429] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1507879413631_0001_m_000000_0 is : 0.0
2017-10-13 12:30:41,870 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1507879413631_0001_m_000000_0: AttemptID:attempt_1507879413631_0001_m_000000_0 Timed out after 10 secs
2017-10-13 12:30:41,871 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1507879413631_0001_m_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP
2017-10-13 12:30:41,872 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_1507879413631_0001_01_000002 taskAttempt attempt_1507879413631_0001_m_000000_0
2017-10-13 12:30:41,873 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1507879413631_0001_m_000000_0
2017-10-13 12:30:41,873 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : node1:39015
2017-10-13 12:30:41,896 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1507879413631_0001_m_000000_0 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP
2017-10-13 12:30:41,896 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT
2017-10-13 12:30:41,897 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1507879413631_0001_m_000000_0 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED
2017-10-13 12:30:41,902 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved node3 to /default-rack
2017-10-13 12:30:41,903 INFO [Thread-52] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: 1 failures on node node1
2017-10-13 12:30:41,904 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1507879413631_0001_m_000000_1 TaskAttempt Transitioned from NEW to UNASSIGNED
2017-10-13 12:30:41,905 INFO [Thread-52] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added attempt_1507879413631_0001_m_000000_1 to list of failed maps
2017-10-13 12:30:42,847 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:1
2017-10-13 12:30:42,865 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1507879413631_0001: ask=1 release= 0 newContainers=0 finishedContainers=1 resourcelimit=<memory:7168, vCores:1> knownNMs=3
2017-10-13 12:30:42,865 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1507879413631_0001_01_000002
2017-10-13 12:30:42,866 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:1
2017-10-13 12:30:42,866 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1507879413631_0001_m_000000_0: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
问题出在哪里。我检查了所有日志，但没有上面的错误消息。第二，当我在独立机器（伪模式）上运行这个工作时，它完成没有任何问题。
答案 0 :(得分：0)
您可以在日志中看到您收到以下错误：
AttemptID:attempt_1507879413631_0001_m_000000_0 Timed out after 10 secs
您可以通过修改此处的设置来尝试增加超时：
<property>
<name>mapreduce.task.timeout</name>
<value>10000</value>
<description>The number of milliseconds before a task will be
terminated if it neither reads an input, writes an output, nor
updates its status string. A value of 0 disables the timeout.
</description>
</property>
您的设置为10秒，通常使用超时600（10分钟）。所以你可以把它改成600000。