Question

我已经等了很久才看到减速机阶段开始，但它没有开始。我已经尝试格式化namenode但仍然是同样的问题。我必须在减速器必须运行的任何地方遇到这个问题。在这方面，任何人都可以帮助我。

hive> LOAD DATA LOCAL INPATH '/home/training/Desktop/student_performs_CSV.csv' OVERWRITE INTO TABLE stud_marks;
Copying data from file:/home/training/Desktop/student_performs_CSV.csv
Copying file: file:/home/training/Desktop/student_performs_CSV.csv
Loading data to table default.stud_marks
Deleted hdfs://localhost:9000/user/hive/warehouse/stud_marks
OK
Time taken: 1.633 seconds
hive> select count(*) from stud_marks;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201211141618_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201211141618_0002
Kill Command = /home/training/Installations/hadoop-1.0.3/libexec/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201211141618_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2012-11-14 16:48:29,822 Stage-1 map = 0%,  reduce = 0%
2012-11-14 16:48:42,270 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec
2012-11-14 16:48:43,349 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec
2012-11-14 16:48:44,429 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec
2012-11-14 16:48:45,507 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec
2012-11-14 16:48:46,580 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

..
.
.
.
.
.
.
2012-11-14 16:56:44,394 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 54.9 sec
2012-11-14 16:56:45,417 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 54.9 sec
2012-11-14 16:56:46,449 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 54.9 sec

Answer 1

您只是尝试将数据从本地计算机加载到HIVE表中。在HIVE中，此操作不会启动任何减少阶段，数据加载发生在映射阶段，并且在减少阶段不会执行任何操作。

要查找一些reduce任务，请尝试使用“group by”等操作进行其他一些HIVE查询。

Answer 2

通过修改＆＃39; / etc / hosts＆＃39;中的IP来解决问题。匹配

ifconfig＆gt; inet addr

参数

减速机阶段根本没有启动。总是'reducer = 0％'

2 个答案: