Question

我目前正在使用Hadoop 0.21.0,985326和一个由6个工作节点和一个头节点组成的集群。提交常规mapreduce作业失败了，但我不明白为什么。有没有人见过这个例外？

org.apache.hadoop.mapred.Child: Exception running child : java.io.IOException: Spill failed
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.checkSpillException(MapTask.java:1379)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$200(MapTask.java:711)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
    at java.io.DataOutputStream.write(DataOutputStream.java:90)
    at org.apache.hadoop.io.Text.write(Text.java:290)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:100)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:84)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:967)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:583)
    at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:92)
    at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:111)
    at be.ac.ua.comp.ronny.riki.invertedindex.FilteredInvertedIndexBuilder$Map.map(FilteredInvertedIndexBuilder.java:113)
    at be.ac.ua.comp.ronny.riki.invertedindex.FilteredInvertedIndexBuilder$Map.map(FilteredInvertedIndexBuilder.java:1)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:652)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
    at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.io.ArrayWritable.<init>()
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:123)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:68)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:44)
    at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:145)
    at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)
    at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:291)
    at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:168)
    at org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1432)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1457)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$600(MapTask.java:711)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:1349)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.io.ArrayWritable.<init>()
    at java.lang.Class.getConstructor0(Class.java:2706)
    at java.lang.Class.getDeclaredConstructor(Class.java:1985)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
    ... 10 more

目前，我正在尝试一些配置参数，希望这个错误消失，但直到现在这个都没有成功。我正在调整的配置参数是：

mapred.map.tasks = 60
mapred.reduce.tasks = 12
Job.MAP_OUTPUT_COMPRESS（或mapreduce.map.output.compress）= true
Job.IO_SORT_FACTOR（或mapreduce.task.io.sort.factor）= 10
Job.IO_SORT_MB（或mapreduce.task.io.sort.mb）= 256
Job.MAP_JAVA_OPTS（或mapreduce.map.java.opts）=“ - Xmx256”或“-Xmx512”
Job.REDUCE_JAVA_OPTS（或mapreduce.reduce.java.opts）=“ - Xmx256”或“-Xmx512”

有人可以解释为什么会发生上述异常吗？以及如何避免它？或者只是简单解释一下hadoop溢出操作意味着什么？

Answer 1

好的，所有问题都解决了。

Map-Reduce序列化操作需要实现 org.apache.hadoop.io.ArrayWritable 的默认构造函数。
Hadoops实现没有为ArrayWritable提供默认构造函数这就是java.lang.NoSuchMethodException：org.apache.hadoop.io.ArrayWritable。（）被抛出并导致奇怪的溢出异常的原因。

一个简单的包装器使得ArrayWritable真的可以写并修复它！奇怪的是Hadoop没有提供这个。

Answer 2

当我的一个地图作业的输出产生制表符（“\ t”）或换行符（“\ r”或“\ n”）时，我遇到了这个问题 - Hadoop无法很好地处理这个问题并失败。我能够使用这段Python代码解决这个问题：

if "\t" in output:
  output = output.replace("\t", "")
if "\r" in output:
  output = output.replace("\r", "")
if "\n" in output:
  output = output.replace("\n", "")

您可能需要为自己的应用做些其他事情。

Hadoop Spill失败

2 个答案: