错误" IndexOutOfBoundsException索引:3,大小:3"在运行hive代码时

时间:2016-10-10 05:56:36

标签: hadoop hive

我每天运行下面的代码,每次都成功。但是当我今天运行它时,会出现错误IndexOutOfBoundsException Index:3,Size:3。当我删除子句" where member_srl is not null"时,它成功了。所以我不知道我的代码有什么问题。为什么它今天无法运行。感谢。

select   member_srl, dt, sessionid  , (max(reg_time)-min(reg_time)) as duration, count(reg_time) as click_cnt 
      from
       (
         select cast(member_srl as bigint) as member_srl, reg_date as dt, sessionid, cast(SUBSTRING(reg_time,1,2)*3600+SUBSTRING(reg_time,3,2)*60+SUBSTRING(reg_time,5,2) as bigint) as reg_time
         from default.daily_session 
         where member_srl<>'' and dt = '20161009'
        union all
        select cast(member_srl as bigint) as member_srl, reg_date as dt, sessionid, cast(SUBSTRING(reg_time,1,2)*3600+SUBSTRING(reg_time,3,2)*60+SUBSTRING(reg_time,5,2) as bigint) as reg_time
         from default.daily_session_mobile  
         where member_srl<>'' and dt = '20161009'
        union all 
        select cast(member_srl as bigint) as member_srl, reg_date as dt, sessionid, cast(SUBSTRING(reg_time,1,2)*3600+SUBSTRING(reg_time,3,2)*60+SUBSTRING(reg_time,5,2) as bigint) as reg_time
         from default.daily_session_ios  
         where member_srl<>'' and dt = '20161009'
       ) base  where member_srl is not null 
       group by member_srl, dt, sessionid  

1 个答案:

答案 0 :(得分:0)

这是一个bug,在1.3.0版本中已经解决。

如果您像我一样使用旧版本,那么 set hive.optimize.ppd=false; 可以解决您的问题。(已弃用)

我的版本是 Hive 1.1.0-cdh5.6.0

我将 hive.optimize.ppd 设置为 false,发现对分区字段的过滤不起作用,所有分区表中的分区都会被映射器读取。

终于。

我更改了查询,使用另一个子查询解决了问题。