Question

嘿，架构是这样的：对于整个数据集，我们应该先按machine_id排序，然后再按ss2k排序。之后，对于每台机器，我们应该找到至少连续带有5 flag ='census'的所有行。在此数据集中，结果应为所有黄色行。

我不能通过以下方式返回黄色块的最后4行：

drop table if exists qz_panel_census_228_rank;
create table qz_panel_census_228_rank as
select t.*
from (select t.*,
             count(*) filter (where flag = 'census') over (partition by machine_id, date order by ss2k rows between current row and 4 following) as census_cnt5,
             count(*) filter (where flag = 'census') over (partition by machine_id, date) as count_census,
             row_number() over (partition by machine_id, date order by ss2k) as seqnum,
             count(*) over (partition by machine_id, date) as cnt
      from qz_panel_census_228 t
     ) t
where census_cnt5 = 5 
group by 1,2,3,4,5,6,7,8,9,10,11
DISTRIBUTED BY (machine_id);

Answer 1

您很近，但是您需要双向搜索：

   select t.*
    from (select t.*,
            case when count(*) filter (where flag = 'census')
                      over (partition by machine_id, date
                            order by ss2k
                            rows between 4 preceding and current row) = 5 
                   or count(*) filter (where flag = 'census') 
                      over (partition by machine_id, date
                            order by ss2k
                            rows between current row and 4 following) = 5
                 then 1
                 else 0
             end  as flag
          from qz_panel_census_228 t
         ) t
    where flag = 1

编辑：

该方法将不起作用，除非您为每个可能的5行窗口添加一个额外的计数，例如3个在前和1个在后，2个在前和2个在后，依此类推。这会导致代码很丑陋，而且不太灵活。

解决此空白和孤岛问题的常用方法是先将连续的行分配给一个公共组：

    select *
    from 
     ( 
        select t2.*,
           count(*) over (partition by machine_id, date, grp) as cnt
        from
          (
           select t1.*
            from (select t.*,
                    -- keep the same number for 'census' rows
                    sum(case when flag = 'census' then 0 else 1 end)
                    over (partition by machine_id, date
                          order by ss2k
                          rows unbounded preceding) as grp
                  from qz_panel_census_228 t
          ) t1
         where flag = 'census' -- only census rows
      ) as t2
) t3
     where cnt >= 5  -- only groups of at least 5 census rows

Answer 2

哇，必须有一个更好的方法来做到这一点，但是我唯一能想到的方法就是创建连续的“普查”值块。这看起来很糟糕，但可能会催生一个更好的主意。

with q1 as (
  select
    machine_id, recorded, ss2k, flag, date,
    case
      when flag = 'census' and
        lag (flag) over (order by machine_id, ss2k) != 'census'
          then 1
      else 0
    end as block
  from foo
),
q2 as (
  select
    machine_id, recorded, ss2k, flag, date,
    sum (block) over (order by machine_id, ss2k) as group_id,
    case when flag = 'census' then 1 else 0 end as census
  from q1
),
q3 as (
  select 
    machine_id, recorded, ss2k, flag, date, group_id,
    sum (census) over (partition by group_id order by ss2k) as max_count
  from q2
),
groups as (
  select group_id
  from q3
  group by group_id
  having max (max_count) >= 5
)
select
  q2.machine_id, q2.recorded, q2.ss2k, q2.flag, q2.date
from
  q2
  join groups g on q2.group_id = g.group_id
where
  q2.flag = 'census'

如果您在with子句中单独运行每个查询，我想您会看到它如何发展的。

如何返回黄色人口普查区中的所有行？

2 个答案: