当我尝试启动简单的单词计数mapreduce示例时,我明白了
[cloudera-scm@ivana-namenode2 /opt/MapReduce/wordcount]$ pydoop script wc.py /user/cloudera-scm/Mapduce/wordcount/data/text /user/cloudera-scm/MapReduce/wordcount/output
18/08/14 14:11:49 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files
18/08/14 14:11:49 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
18/08/14 14:11:49 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
18/08/14 14:11:49 INFO Configuration.deprecation: hadoop.pipes.java.recordreader is deprecated. Instead, use mapreduce.pipes.isjavarecordreader
18/08/14 14:11:49 INFO Configuration.deprecation: mapred.cache.archives is deprecated. Instead, use mapreduce.job.cache.archives
18/08/14 14:11:49 INFO Configuration.deprecation: hadoop.pipes.java.recordwriter is deprecated. Instead, use mapreduce.pipes.isjavarecordwriter
18/08/14 14:11:49 INFO Configuration.deprecation: mapred.textoutputformat.separator is deprecated. Instead, use mapreduce.output.textoutputformat.separator
18/08/14 14:11:49 INFO Configuration.deprecation: mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress
18/08/14 14:11:50 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm59
18/08/14 14:11:51 WARN mapreduce.JobResourceUploader: No job jar file set. User classes may not be found. See Job or Job#setJar(String).
18/08/14 14:11:51 INFO input.FileInputFormat: Total input paths to process : 1
18/08/14 14:11:52 INFO mapreduce.JobSubmitter: number of splits:1
18/08/14 14:11:52 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1534248543348_0007
18/08/14 14:11:52 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
18/08/14 14:11:52 INFO impl.YarnClientImpl: Submitted application application_1534248543348_0007
18/08/14 14:11:52 INFO mapreduce.Job: The url to track the job: http://ivana-namenode2:8088/proxy/application_1534248543348_0007/
18/08/14 14:11:52 INFO mapreduce.Job: Running job: job_1534248543348_0007
18/08/14 14:15:07 INFO mapreduce.Job: Job job_1534248543348_0007 running in uber mode : false
18/08/14 14:15:07 INFO mapreduce.Job: map 0% reduce 0%
18/08/14 14:15:07 INFO mapreduce.Job: Job job_1534248543348_0007 failed with state KILLED due to: Application killed by user.
18/08/14 14:15:07 INFO mapreduce.Job: Counters: 0
18/08/14 14:15:07 INFO util.ExitUtil: Exiting with status 1
Traceback (most recent call last):
File "/usr/local/bin/pydoop", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/site-packages/pydoop/app/main.py", line 81, in main
args.func(args, unknown)
File "/usr/local/lib/python2.7/site-packages/pydoop/app/script.py", line 130, in run
scripter.run()
File "/usr/local/lib/python2.7/site-packages/pydoop/app/script.py", line 119, in run
submitter.run()
File "/usr/local/lib/python2.7/site-packages/pydoop/app/submit.py", line 376, in run
logger=self.logger, keep_streams=False)
File "/usr/local/lib/python2.7/site-packages/pydoop/hadut.py", line 302, in run_class
keep_streams=keep_streams)
File "/usr/local/lib/python2.7/site-packages/pydoop/hadut.py", line 167, in run_cmd
raise RunCmdError(p.returncode, ' '.join(_args), error)
pydoop.hadut.RunCmdError: command exited with 1 status
另外,当我尝试启动jar示例时,我遇到了同样的问题,因此与代码无关。
下面,您具有指向我的main.py,script.py和Submit.py的链接。我不能将它们放在这里,因为堆栈溢出最多可容纳30000个字符
https://www.dropbox.com/sh/1mabmgfywa38xib/AAB946SUK8-A8VfWfieQG_4ha?dl=0