Question

如果我在我的CDH3集群上运行MapReduce任务，初始化多个Mapper，第一个Mapper永远不会完成。我无法在任何日志文件中发现错误。我得到的唯一相关日志是：

Task xxx failed to report status for 1800 seconds. Killing!

在几个任务被杀死后，Job失败并抛出此异常：

2012-01-30 11:30:00,998 ERROR server jobsubd PlanRunner saw exception. Traceback (most recent call last): File "/usr/share/hue/apps/jobsub/src/jobsub/server.py", line 160, in run self.run_bin_hadoop_step(step.bin_hadoop_step) File "/usr/share/hue/apps/jobsub/src/jobsub/server.py", line 292, in run_bin_hadoop_step raise Exception("bin/hadoop returned non-zero %d" % retcode) Exception: bin/hadoop returned non-zero 1 2012-01-30 11:30:01,027 INFO server Marked jobsubd job 130 as done.

有人可以给我一些关于在哪里寻找可能解决方案的提示。

谢谢！

Mapper：

@Override
public void map(LongWritable offset, Text line, Context context)
            throws IOException {
    try {
            DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
            DocumentBuilder builder = factory.newDocumentBuilder();
            Document xml = builder.parse(new ByteArrayInputStream(line.getBytes()));

            Node root = xml.getFirstChild();
            NodeList nodes = root.getChildNodes();

            for (int indexElement = 0; indexElement < nodes.getLength(); indexElement++) {
                Element element = (Element) nodes.item(indexElement);
                Put put = new Put(Bytes.toBytes(indexElement));

                if (element.hasAttributes()) {
                    NamedNodeMap attributes = element.getAttributes();
                    for (int indexAttribute = 0; indexAttribute < attributes
                            .getLength(); indexAttribute++) {
                        Attr attribuut = (Attr) attributes.item(indexAttribute);
                        put.add(Bytes.toBytes("attributes"),
                                Bytes.toBytes(attribuut.getNodeName()),
                                Bytes.toBytes(attribuut.getNodeValue()));
                    }
                }
                if (element.getNodeValue() != null) {
                    put.add(Bytes.toBytes("value"), Bytes.toBytes(element.getNodeName()), Bytes.toBytes(element.getNodeValue()));
                }
                context.write(new ImmutableBytesWritable() , put);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

输入是一个XML文件。其他Mappers，例如JSON导入器，也存在同样的问题。

Hadoop MapReduce不会运行第一个Mapper

0 个答案: