Hive:选择前一个第n行,列的值最小

时间:2016-06-14 12:00:14

标签: hadoop hive hiveql

我有这样的数据

ID  START_DATE STATUS
10  2013-05-29 FREE
10  2013-05-29 PAID
10  2014-05-30 PAID
10  2014-11-29 FREE
10  2014-12-02 PAID
10  2015-09-29 PAID
10  2015-12-02 PAID
10  2016-04-04 PAID
10  2016-04-05 FREE

我的输出应该只包含status =" FREE"的行。每次状态为FREE时,我都需要获取Status =" PAID"的前一个最小日期。

ID STATUS PREVIOUS_MIN_PAID_START_DATE
10 FREE   NULL
10 FREE   2013-05-29
10 FREE   2014-12-02

LAG()函数仅提供前一个第一个值,如何获得先前的最小值(第n个)?

SELECT
  ID,
  STATUS,
  LAG(CASE WHEN STATUS = 'PAID' THEN  START_DATE, 1)  
     OVER (PARTITION BY ID ORDER BY START_DATE) AS previous_paid_start_date
FROM
  TEMP
WHERE
  STATUS = 'FREE'

1 个答案:

答案 0 :(得分:0)

不确定你为什么收到downvotes,我认为这是一个非常有趣(并且描述得很好的问题)。无论如何,这是 一种方式 来做到这一点,虽然我必须承认它感觉次优和黑客。

基本上,你需要的是一个创建一组"当前" FREE以及随后的所有PAID,直到您到达下一个FREE(我希望我已正确理解这一点)。举例说明:

id  start_date status idx
10  2013-05-29 FREE   0
10  2013-05-29 PAID   1 
10  2014-05-30 PAID   1
10  2014-11-29 FREE   1
10  2014-12-02 PAID   2
10  2015-09-29 PAID   2
10  2015-12-02 PAID   2
10  2016-04-04 PAID   2
10  2016-04-05 FREE   2

然后从那里你可以获得最小的start_date,其中状态为PAID在id窗口和新创建的索引上。

<强>查询

WITH tmp_table AS (
  SELECT *
    , SUM(flg) OVER (PARTITION BY id ROWS UNBOUNDED PRECEDING) AS s
  FROM (
    SELECT *
      , LEAD(CASE WHEN status='FREE' THEN 1 ELSE 0 END, 1, 0) OVER (PARTITION BY id) AS flg
    FROM database.original_table) x )

SELECT id
  , status
  , prev_date
FROM (
  SELECT t.id, t.status, t.s, b.prev_date
  FROM tmp_table t
  LEFT OUTER JOIN (
    SELECT id, s, MIN(start_date) AS prev_date
    FROM tmp_table
    WHERE status='PAID'
    GROUP BY id, s ) b
  ON b.id=t.id AND b.s=t.s ) f
WHERE status='FREE'

<强>输出

id status prev_date
10 FREE   NULL
10 FREE   2013-05-29
10 FREE   2014-12-02