Hadoop MapReduce作业挂起

时间:2015-02-27 18:18:54

标签: mapreduce hadoop2

我正在尝试在我的Ubuntu桌面上使用最新的Hadoop版本2.6.0,Java SDK 1.70来模拟Hadoop环境。我为hadoop配置了必要的环境参数,并且所有进程都已启动并运行,可以使用以下jps命令查看它们:

nandu@nandu-Desktop:~$ jps
2810 NameNode
3149 SecondaryNameNode
3416 NodeManager
3292 ResourceManager
2966 DataNode
4805 Jps

我还可以通过Firefox浏览器看到上述信息以及dfs文件。但是,当我尝试运行一个简单的WordCound MapReduce作业时,它会挂起并且不会产生任何输出或显示任何错误消息。过了一会儿,我使用“hadoop job -kill”命令杀死了进程。你能指导我,找到这个问题的原因以及如何解决它?我在作业开始之下并杀死(结束)截图。

如果您需要其他信息,请告诉我们。

我们非常感谢您的帮助。

谢谢,

===================================================================
nandu@nandu-Desktop:~/dev$ hadoop jar wc.jar WordCount /user/nandu/input /user/nandu/output
15/02/27 10:35:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/02/27 10:35:20 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/02/27 10:35:21 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
15/02/27 10:35:21 INFO input.FileInputFormat: Total input paths to process : 2
15/02/27 10:35:21 INFO mapreduce.JobSubmitter: number of splits:2
15/02/27 10:35:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1425048764581_0003
15/02/27 10:35:22 INFO impl.YarnClientImpl: Submitted application application_1425048764581_0003
15/02/27 10:35:22 INFO mapreduce.Job: The url to track the job: http://nandu-Desktop:8088/proxy/application_1425048764581_0003/
15/02/27 10:35:22 INFO mapreduce.Job: Running job: job_1425048764581_0003

==================== at this point the job was killed ===================

15/02/27 10:38:23 INFO mapreduce.Job: Job job_1425048764581_0003 running in uber mode : false
15/02/27 10:38:23 INFO mapreduce.Job:  map 0% reduce 0%
15/02/27 10:38:23 INFO mapreduce.Job: Job job_1425048764581_0003 failed with state KILLED due to: Application killed by user.
15/02/27 10:38:23 INFO mapreduce.Job: Counters: 0

1 个答案:

答案 0 :(得分:3)

我在hadoop包中运行提供的MapReduce示例时遇到了类似的问题。在我的情况下,由于我的VM上的磁盘空间不足(大约1.5 GB是空的),它挂起了。当我释放一些磁盘空间时,它运行得非常好。另外,请检查是否满足其他系统资源要求。