我正在尝试在我的Ubuntu桌面上使用最新的Hadoop版本2.6.0,Java SDK 1.70来模拟Hadoop环境。我为hadoop配置了必要的环境参数,并且所有进程都已启动并运行,可以使用以下jps命令查看它们:
nandu@nandu-Desktop:~$ jps 2810 NameNode 3149 SecondaryNameNode 3416 NodeManager 3292 ResourceManager 2966 DataNode 4805 Jps
我还可以通过Firefox浏览器看到上述信息以及dfs文件。但是,当我尝试运行一个简单的WordCound MapReduce作业时,它会挂起并且不会产生任何输出或显示任何错误消息。过了一会儿,我使用“hadoop job -kill”命令杀死了进程。你能指导我,找到这个问题的原因以及如何解决它?我在作业开始之下并杀死(结束)截图。
如果您需要其他信息,请告诉我们。
我们非常感谢您的帮助。
谢谢,
=================================================================== nandu@nandu-Desktop:~/dev$ hadoop jar wc.jar WordCount /user/nandu/input /user/nandu/output 15/02/27 10:35:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 15/02/27 10:35:20 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 15/02/27 10:35:21 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 15/02/27 10:35:21 INFO input.FileInputFormat: Total input paths to process : 2 15/02/27 10:35:21 INFO mapreduce.JobSubmitter: number of splits:2 15/02/27 10:35:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1425048764581_0003 15/02/27 10:35:22 INFO impl.YarnClientImpl: Submitted application application_1425048764581_0003 15/02/27 10:35:22 INFO mapreduce.Job: The url to track the job: http://nandu-Desktop:8088/proxy/application_1425048764581_0003/ 15/02/27 10:35:22 INFO mapreduce.Job: Running job: job_1425048764581_0003 ==================== at this point the job was killed =================== 15/02/27 10:38:23 INFO mapreduce.Job: Job job_1425048764581_0003 running in uber mode : false 15/02/27 10:38:23 INFO mapreduce.Job: map 0% reduce 0% 15/02/27 10:38:23 INFO mapreduce.Job: Job job_1425048764581_0003 failed with state KILLED due to: Application killed by user. 15/02/27 10:38:23 INFO mapreduce.Job: Counters: 0
答案 0 :(得分:3)
我在hadoop包中运行提供的MapReduce
示例时遇到了类似的问题。在我的情况下,由于我的VM上的磁盘空间不足(大约1.5 GB是空的),它挂起了。当我释放一些磁盘空间时,它运行得非常好。另外,请检查是否满足其他系统资源要求。