由于任务失败,作业 mapreduce 失败

时间:2021-01-04 15:37:28

标签: hadoop giraph aggregator

我是 giraph 的新手,最近我正在尝试了解聚合器。我从以下链接运行 PageRank 程序

http://buttercola.blogspot.com/2015/04/giraph-page-rank.html

但是我在执行时遇到错误。 mapReduce 的工作并没有一路走好。 以下是错误消息:

bigdata@alpha:~/Programs/hadoop$ bin/hadoop jar ./share/hadoop/common/giraph-core.jar org.apache.giraph.GiraphRunner example1.MyPageRankComputationWithAggregator -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/bigdata/giraph/graph-data2.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/bigdata/giraph/MPRCWA-data2 -w 1 -ca giraph.SplitMasterWorker=false

21/01/04 21:57:09 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
21/01/04 21:57:09 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
21/01/04 21:57:09 INFO utils.ConfigurationUtils: Setting custom argument [giraph.SplitMasterWorker] to [false] in GiraphConfiguration
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/bigdata/Programs/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/bigdata/Programs/hadoop/share/hadoop/common/giraph-core.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/01/04 21:57:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
21/01/04 21:57:16 INFO Configuration.deprecation: mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max
21/01/04 21:57:16 INFO Configuration.deprecation: mapred.job.map.memory.mb is deprecated. Instead, use mapreduce.map.memory.mb
21/01/04 21:57:16 INFO Configuration.deprecation: mapred.job.reduce.memory.mb is deprecated. Instead, use mapreduce.reduce.memory.mb
21/01/04 21:57:16 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
21/01/04 21:57:17 INFO Configuration.deprecation: mapreduce.user.classpath.first is deprecated. Instead, use mapreduce.job.user.classpath.first
21/01/04 21:57:17 INFO Configuration.deprecation: mapred.map.max.attempts is deprecated. Instead, use mapreduce.map.maxattempts
21/01/04 21:57:17 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
21/01/04 21:57:17 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
21/01/04 21:57:17 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
21/01/04 21:57:27 INFO mapreduce.JobSubmitter: number of splits:1
21/01/04 21:57:28 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1609768211026_0001
21/01/04 21:57:31 INFO impl.YarnClientImpl: Submitted application application_1609768211026_0001
21/01/04 21:57:32 INFO mapreduce.Job: The url to track the job: http://alpha:8088/proxy/application_1609768211026_0001/
21/01/04 21:57:32 INFO job.GiraphJob: Tracking URL: http://alpha:8088/proxy/application_1609768211026_0001/
21/01/04 21:57:32 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 2 mappers
21/01/04 21:59:05 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer alpha:22181 --zkNode /_hadoopBsp/job_1609768211026_0001/_haltComputation'
21/01/04 21:59:05 INFO mapreduce.Job: Running job: job_1609768211026_0001
21/01/04 21:59:06 INFO mapreduce.Job: Job job_1609768211026_0001 running in uber mode : false
21/01/04 21:59:06 INFO mapreduce.Job:  map 100% reduce 0%
21/01/04 22:14:26 INFO mapreduce.Job: Job job_1609768211026_0001 failed with state FAILED due to: Task failed task_1609768211026_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

21/01/04 22:14:26 INFO mapreduce.Job: Counters: 8
    Job Counters 
        Failed map tasks=1
        Launched map tasks=1
        Other local map tasks=1
        Total time spent by all maps in occupied slots (ms)=953661
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=953661
        Total vcore-milliseconds taken by all map tasks=953661
        Total megabyte-milliseconds taken by all map tasks=976548864

0 个答案:

没有答案
相关问题