Hive Over子句同时运行多个reducers

时间:2017-04-28 01:43:13

标签: hadoop hive

如何运行/激活此类脚本以同时启动所有Reducer?

`SELECT id, map(
 'r02', string(PERCENT_RANK() OVER (ORDER BY maps['k02']))
, 'r03', string(PERCENT_RANK() OVER (ORDER BY maps['k03']))
, 'r04', string(PERCENT_RANK() OVER (ORDER BY maps['k04']))
, 'r06', string(PERCENT_RANK() OVER (ORDER BY maps['k06']))
, 'r09', string(PERCENT_RANK() OVER (ORDER BY maps['k09']))
, 'r10', string(PERCENT_RANK() OVER (ORDER BY maps['k10']))
, 'r11', string(PERCENT_RANK() OVER (ORDER BY maps['k11'])))
FROM database

映射完成后,似乎一次只运行一个reducer

INFO  : Map 1: 3/3      Reducer 2: 1(+1)/2      Reducer 3: 0/2  Reducer 4: 0/2  Reducer 5: 0/2  Reducer 6: 0/2  Reducer 7: 0/2Reducer 8: 0/2

INFO  : Map 1: 3/3      Reducer 2: 2/2  Reducer 3: 1(+1)/2      Reducer 4: 0/2  Reducer 5: 0/2  Reducer 6: 0/2  Reducer 7: 0/2Reducer 8: 0/2

如何更改脚本以运行并运行并行的所有Reducer?谢谢,全部。

1 个答案:

答案 0 :(得分:0)

请尝试 -

set hive.exec.parallel=true;