Question

我正在浏览Apache Giraph快速入门教程：http://giraph.apache.org/quick_start.html并成功设置了伪分布式hadoop集群，并成功运行了示例mapreduce作业。

然而，当我移动到Giraph部分时，我已经使用maven安装了Giraph，并且由于某种原因我无法运行最简单的Giraph工作。

我已将tiny_graph添加到我的hdfs中，但运行了他们的示例命令：

hduser@fonz-VirtualBox:~/Giraph$ $HADOOP_HOME/bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths -w 1

给我：

14/12/18 11:19:27 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
14/12/18 11:19:27 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
14/12/18 11:19:28 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 1, old value = 4)
14/12/18 11:19:31 INFO job.GiraphJob: Tracking URL: http://hdnode01:50030/jobdetails.jsp?jobid=job_201412171202_0010
14/12/18 11:19:31 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 2 mappers
14/12/18 11:20:59 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer fonz-virtualbox:22181 --zkNode /_hadoopBsp/job_201412171202_0010/_haltComputation'
14/12/18 11:20:59 INFO mapred.JobClient: Running job: job_201412171202_0010
14/12/18 11:21:00 INFO mapred.JobClient:  map 100% reduce 0%
14/12/18 11:30:16 INFO mapred.JobClient:  map 50% reduce 0%
14/12/18 11:30:24 INFO mapred.JobClient: Job complete: job_201412171202_0010
14/12/18 11:30:24 INFO mapred.JobClient: Counters: 6
14/12/18 11:30:24 INFO mapred.JobClient:   Job Counters 
14/12/18 11:30:24 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=1244789
14/12/18 11:30:24 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/12/18 11:30:24 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/12/18 11:30:24 INFO mapred.JobClient:     Launched map tasks=2
14/12/18 11:30:24 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
14/12/18 11:30:24 INFO mapred.JobClient:     Failed map tasks=1

非常感谢任何有关失败原因的想法！

非常感谢。

威尔

Answer 1

最短路径示例没有主方法，它使用硬编码源ID作为1，以启动算法。因此，在100％Map之后没有投票停止。转到https://www.youtube.com/watch?v=yEHeJhMXJy0，更新sssp.java文件并将其导出到新的Jar。然后运行它。我遇到了同样的问题并修复了它。

Answer 2

下载Hadoop时，下载hadoop-0.20.205.0.tar.gz而不是 hadoop-0.20.203.0rc1.tar.gz以避免Failed map task = 1错误。

Giraph教程ShortestPath示例作业失败

2 个答案: