Question

我是Hadoop的新手，我正在使用单个节点进行POC。这是第二次发生，不知道如何解决。使用Ctrl + C停止地图缩小后会发生这种情况。 SQLServer正在通过Sqoop将数据导入HBase，并且在出现错误和Ctrl + C之后，nunhum另一个工作正常工作，卡在ACCEPTED中。已经排除了所有工作申请。我重新启动了整个服务器。资源管理器已相应配置（10Gb内存，2个内核），但不知道会发生什么。

[root@hadoop01 /]# hadoop version
Hadoop 2.6.0.2.2.6.0-2800
Subversion git@github.com:hortonworks/hadoop.git -r acb70ecfae2c3c5ab46e24b0caebceaec16fdcd0
Compiled by jenkins on 2015-05-18T20:21Z
Compiled with protoc 2.5.0
From source with checksum a25c30f622eb057f47e2155f78dba5e
This command was run using /usr/hdp/2.2.6.0-2800/hadoop/hadoop-common-2.6.0.2.2.6.0-2800.jar

[root@hadoop01 fausto.branco]# cat /etc/centos-release
CentOS release 6.6 (Final)

sudo -u hdfs sqoop import –connect “jdbc:sqlserver://SQLServerIP:1433;database=db_POC;username=hdpteste;password=xxxxxx” \
–hbase-create-table \
–hbase-table vw_hdp_Arquivo \
–hbase-row-key “id_Arquivo, Valor” \
–column-family cf_name \
–driver com.microsoft.sqlserver.jdbc.SQLServerDriver \
–table “dbo.vw_hdp_Arquivo” -m 1

15/07/14 19:43:51 INFO zookeeper.ZooKeeper: Session: 0x14e8e1385000001 closed
15/07/14 19:43:51 INFO zookeeper.ClientCnxn: EventThread shut down
15/07/14 19:43:51 INFO impl.TimelineClientImpl: Timeline service address: hxxp://hadoop01.POC.local:8188/ws/v1/timeline/
15/07/14 19:43:51 INFO client.RMProxy: Connecting to ResourceManager at hadoop01.POC.local/XXX.XXX.XXX.XXX:8050
15/07/14 19:43:53 INFO db.DBInputFormat: Using read commited transaction isolation
15/07/14 19:43:53 INFO mapreduce.JobSubmitter: number of splits:1
15/07/14 19:43:54 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1436900603357_0002
15/07/14 19:43:54 INFO impl.YarnClientImpl: Submitted application application_1436900603357_0002
15/07/14 19:43:54 INFO mapreduce.Job: The url to track the job: hxxp://hadoop01.POC.local:8088/proxy/application_1436900603357_0002/
15/07/14 19:43:54 INFO mapreduce.Job: Running job: job_1436900603357_0002

[root@hadoop01 fausto.branco]# yarn application -status application_1436900603357_0002
15/07/14 19:45:05 INFO impl.TimelineClientImpl: Timeline service address: hxxp://hadoop01.POC.local:8188/ws/v1/timeline/
15/07/14 19:45:06 INFO client.RMProxy: Connecting to ResourceManager at hadoop01.POC.local/XXX.XXX.XXX.XXX:8050
Application Report :
Application-Id : application_1436900603357_0002
Application-Name : dbo.vw_hdp_Arquivo.jar
Application-Type : MAPREDUCE
User : hdfs
Queue : default
Start-Time : 1436903034500
Finish-Time : 0
Progress : 0%
State : ACCEPTED
Final-State : UNDEFINED
Tracking-URL : N/A
RPC Port : -1
AM Host : N/A
Aggregate Resource Allocation : 0 MB-seconds, 0 vcore-seconds
Diagnostics :

[root@hadoop01 fausto.branco]# hadoop job -status job_1436900603357_0002
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.

15/07/14 19:46:19 INFO impl.TimelineClientImpl: Timeline service address: hxxp://hadoop01.POC.local:8188/ws/v1/timeline/
15/07/14 19:46:19 INFO client.RMProxy: Connecting to ResourceManager at hadoop01.POC.local/XXX.XXX.XXX.XXX:8050

Job: job_1436900603357_0002
Job File: /user/hdfs/.staging/job_1436900603357_0002/job.xml
Job Tracking URL : hxxp://hadoop01.POC.local:8088/proxy/application_1436900603357_0002/
Uber job : false
Number of maps: 0
Number of reduces: 0
map() completion: 0.0
reduce() completion: 0.0
Job state: PREP
retired: false
reason for failure:
Counters: 0

Answer 1

实测！

分析这个项目：聚合资源分配：0 MB-seconds，0 vcore-seconds

我看到节点状态是不健康的，并在日志中找到此消息：

健康报告：1/1本地dirs不好：/ hadoop / yarn / local; 1/1 local-dirs很糟糕：/ hadoop / yarn / log;

所以我放了更多的磁盘并且工作了

Ctrl + C =卡住ACCEPTED后的纱线+ Sqoop

1 个答案: