Question

我有一个汇总到小时级别YYYYMMDDHH的表。数据由外部进程聚合和加载（我无法控制）。我想每月测试一次数据。

我要回答的问题是：一个月中的每个小时都存在吗？

我希望产生的输出将在小时存在的情况下返回1，在小时不存在的情况下返回0。

聚合表看起来像这样...

YYYYMM  YYYYMMDD    YYYYMMDDHH  DATA_AGG
201911  20191101    2019110100  100
201911  20191101    2019110101  125
201911  20191101    2019110103  135
201911  20191101    2019110105  95
…   …   …   …
201911  20191130    2019113020  100
201911  20191130    2019113021  110
201911  20191130    2019113022  125
201911  20191130    2019113023  135

并定义为...

CREATE TABLE YYYYMMDDHH_DATA_AGG AS (
    YYYYMM      VARCHAR,
    YYYYMMDD    VARCHAR,
    YYYYMMDDHH  VARCHAR,
    DATA_AGG    INT
);

我希望在下面产生以下内容...

YYYYMMDDHH     HOUR_EXISTS
2019110100     1
2019110101     1
2019110102     0
2019110103     1
2019110104     0
2019110105     1
...            ...

在上面的示例中，2019110102和2019110104两个小时不存在。

我假设我必须将聚合表与包含所有YYYYMMDDHH组合的计算表结合起来？？

数据库是Snowflake，但假定大多数通用ANSI SQL查询都可以使用。

Answer 1

您可以通过递归CTE获得所需的东西

递归CTE生成可能的小时数列表。然后，如果您有任何与该小时匹配的记录，则简单的左外部联接将为您提供标志。

WITH RECURSIVE CTE (YYYYMMDDHH) as
(
SELECT YYYYMMDDHH
FROM YYYYMMDDHH_DATA_AGG
WHERE YYYYMMDDHH = (SELECT MIN(YYYYMMDDHH) FROM YYYYMMDDHH_DATA_AGG)

UNION ALL 

SELECT TO_VARCHAR(DATEADD(HOUR, 1, TO_TIMESTAMP(C.YYYYMMDDHH, 'YYYYMMDDHH')), 'YYYYMMDDHH') YYYYMMDDHH
FROM CTE C
WHERE  TO_VARCHAR(DATEADD(HOUR, 1, TO_TIMESTAMP(C.YYYYMMDDHH, 'YYYYMMDDHH')), 'YYYYMMDDHH') <= (SELECT MAX(YYYYMMDDHH) FROM YYYYMMDDHH_DATA_AGG)
)

SELECT 
    C.YYYYMMDDHH,
    IFF(A.YYYYMMDDHH IS NOT NULL, 1, 0) HOUR_EXISTS
FROM CTE C
LEFT OUTER JOIN YYYYMMDDHH_DATA_AGG A
    ON C.YYYYMMDDHH = A.YYYYMMDDHH;

如果您的时间范围太长，则cte递归过多会遇到问题。您可以使用所有可能的小时数来创建表或临时表。例如：

CREATE OR REPLACE TEMPORARY TABLE HOURS (YYYYMMDDHH VARCHAR) AS
SELECT TO_VARCHAR(DATEADD(HOUR, SEQ4(), TO_TIMESTAMP((SELECT MIN(YYYYMMDDHH) FROM YYYYMMDDHH_DATA_AGG), 'YYYYMMDDHH')), 'YYYYMMDDHH')
  FROM TABLE(GENERATOR(ROWCOUNT => 10000)) V 
  ORDER BY 1;

SELECT 
    H.YYYYMMDDHH,
    IFF(A.YYYYMMDDHH IS NOT NULL, 1, 0) HOUR_EXISTS
FROM HOURS H
LEFT OUTER JOIN YYYYMMDDHH_DATA_AGG A
    ON H.YYYYMMDDHH = A.YYYYMMDDHH
WHERE H.YYYYMMDDHH <= (SELECT MAX(YYYYMMDDHH) FROM YYYYMMDDHH_DATA_AGG);

然后您可以摆弄发电机数量以确保您有足够的时间。

Answer 2

您可以使用每月的每个小时生成一个表，并LEFT OUTER JOIN对其进行汇总：

WITH EVERY_HOUR AS (
  SELECT TO_CHAR(DATEADD(HOUR, HH, TO_DATE(YYYYMM::TEXT, 'YYYYMM')),
                 'YYYYMMDDHH')::NUMBER YYYYMMDDHH
  FROM (SELECT DISTINCT YYYYMM FROM YYYYMMDDHH_DATA_AGG) t
  CROSS JOIN (
    SELECT ROW_NUMBER() OVER (ORDER BY NULL) - 1 HH
    FROM TABLE(GENERATOR(ROWCOUNT => 745))
  ) h
  QUALIFY YYYYMMDDHH < (YYYYMM + 1) * 10000
)
SELECT h.YYYYMMDDHH, NVL2(a.YYYYMM, 1, 0) HOUR_EXISTS
FROM EVERY_HOUR h
LEFT OUTER JOIN YYYYMMDDHH_DATA_AGG a ON a.YYYYMMDDHH = h.YYYYMMDDHH

Answer 3

这可能有助于您入门。我猜您想使用“合成” [YYYYMMDD]值吗？否则，如果该值不存在，则它们不应出现在列表中

如果存在#_小时，则删除表如果存在#_temp

，则删除表

--Populate a table with hours ranging from 00 to 23
CREATE TABLE #_hours ([hour_value] VARCHAR(2))
DECLARE @_i INT = 0
WHILE (@_i < 24)
    BEGIN
        INSERT INTO #_hours
        SELECT FORMAT(@_i, '0#')
        SET @_i += 1
    END

-- Replicate OP's sample data set
CREATE TABLE #_temp (
    [YYYYMM] INTEGER
    ,   [YYYYMMDD] INTEGER
    ,   [YYYYMMDDHH] INTEGER
    ,   [DATA_AGG] INTEGER
)
INSERT INTO #_temp
VALUES 
(201911, 20191101, 2019110100, 100),
(201911, 20191101, 2019110101, 125),
(201911, 20191101, 2019110103, 135),
(201911, 20191101, 2019110105, 95),
(201911, 20191130, 2019113020, 100),
(201911, 20191130, 2019113021, 110),
(201911, 20191130, 2019113022, 125),
(201911, 20191130, 2019113023, 135)



SELECT X.YYYYMM, X.YYYYMMDD, X.YYYYMMDDHH
    -- Case: If 'target_hours' doesn't exist, then 0, else 1
,   CASE WHEN X.target_hours IS NULL THEN '0' ELSE '1' END AS [HOUR_EXISTS]
FROM (
    -- Select right 2 characters from converted [YYYYMMDDHH] to act as 'target values'
    SELECT T.*
    ,   RIGHT(CAST(T.[YYYYMMDDHH] AS VARCHAR(10)), 2) AS [target_hours]
    FROM #_temp AS T
) AS X
-- Right join to keep all of our hours and only the target hours that match.
RIGHT JOIN #_hours AS H ON H.hour_value = X.target_hours

示例输出：

YYYYMM  YYYYMMDD    YYYYMMDDHH  HOUR_EXISTS
201911  20191101    2019110100  1
201911  20191101    2019110101  1
NULL    NULL        NULL        0
201911  20191101    2019110103  1
NULL    NULL        NULL        0
201911  20191101    2019110105  1
NULL    NULL        NULL        0

Answer 4

使用（几乎）标准sql，您可以将YYYYMMDD的不同值交叉连接到所有可能的小时数列表，然后将其保留到表中：

select concat(d.YYYYMMDD, h.hour) as YYYYMMDDHH,
  case when t.YYYYMMDDHH is null then 0 else 1 end as hour_exists
from (select distinct YYYYMMDD from tablename) as d
cross join (
  select '00' as hour union all select '01' union all
  select '02' union all select '03' union all     
  select '04' union all select '05' union all
  select '06' union all select '07' union all
  select '08' union all select '09' union all
  select '10' union all select '11' union all
  select '12' union all select '13' union all
  select '14' union all select '15' union all
  select '16' union all select '17' union all
  select '18' union all select '19' union all
  select '20' union all select '21' union all
  select '22' union all select '23'
) as h 
left join tablename as t
on concat(d.YYYYMMDD, h.hour) = t.YYYYMMDDHH
order by concat(d.YYYYMMDD, h.hour)

也许在Snowflake中，您可以用一个序列而不是所有那些UNION ALL来轻松构建小时列表。

Answer 5

此版本涵盖了数月和数年的所有天数。这是一组可能的日期与一天中可能的小时数的简单交叉连接-保留到实际日期。

set first = (select min(yyyymmdd::number) from YYYYMMDDHH_DATA_AGG);
set last  = (select max(yyyymmdd::number) from YYYYMMDDHH_DATA_AGG);

with
hours as (select row_number() over (order by null) - 1 h from table(generator(rowcount=>24))),
days as  (
  select 
    row_number() over (order by null) - 1 as n,
    to_date($first::text, 'YYYYMMDD')::date + n as d,
    to_char(d, 'YYYYMMDD') as yyyymmdd
  from table(generator(rowcount=>($last-$first+1)))
)
select days.yyyymmdd || lpad(hours.h,2,0) as YYYYMMDDHH, nvl2(t.yyyymmddhh,1,0) as HOUR_EXISTS
from days cross join hours
left join YYYYMMDDHH_DATA_AGG t on t.yyyymmddhh = days.yyyymmdd || lpad(hours.h,2,0)
order by 1
;

如果愿意，可以将$ first和$ last打包为子查询。

SQL-如果每月存在一个小时，则为“ 1”，如果不存在则为“ 0”，

5 个答案: