Question

我想查找过去30天内有效的唯一身份用户数。我想今天计算这个，但也要过去几天。数据集包含由BigQuery中保存的用户触发的用户ID，日期和事件。通过打开触发事件session_start的移动应用程序，用户处于活动状态。未通过数据集的示例。

| resettable_device_id |     date    |    event      |
------------------------------------------------------
|         xx           |  2017-06-09 | session_start |
|         yy           |  2017-06-09 | session_start |
|         xx           |  2017-06-11 | session_start |
|         zz           |  2017-06-11 | session_start |

我找到了一个适合我的问题的解决方案： BigQuery: how to group and count rows within rolling timestamp window?

到目前为止我的BigQuery脚本：

#standardSQL
WITH daily_aggregation AS (
  SELECT 
    PARSE_DATE("%Y%m%d", event_dim.date) AS day,
    COUNT(DISTINCT user_dim.device_info.resettable_device_id) AS unique_resettable_device_ids
  FROM `ANDROID.app_events_*`,
    UNNEST(event_dim) AS event_dim
  WHERE event_dim.name = "session_start"
  GROUP BY day
)
SELECT 
  day, 
  unique_resettable_device_ids, 
  SUM(unique_resettable_device_ids) 
  OVER(ORDER BY UNIX_SECONDS(TIMESTAMP(day)) DESC ROWS BETWEEN 2592000 PRECEDING AND CURRENT ROW) AS unique_ids_rolling_30_days
FROM daily_aggregation
ORDER BY day

此脚本生成下表：

|      day   | unique_resettable_device_ids | unique_ids_rolling_30_days |
------------------------------------------------------------------------
| 2018-06-05 |            1807              |            2614            |
| 2018-06-06 |             711              |             807            |
| 2018-06-07 |              96              |              96            |

问题是列unique_ids_rolling_30_days只是列unique_resettable_device_ids的累积和。如何在脚本中修复滚动窗口函数？

Answer 1

“问题是，unique_ids_rolling_30_days列只是列unique_resettable_device_ids的累积总和。”

当然，因为那正是代码

SUM(unique_resettable_device_ids) OVER(ORDER BY UNIX_SECONDS(TIMESTAMP(day)) DESC ROWS BETWEEN 2592000 PRECEDING AND CURRENT ROW) AS unique_ids_rolling_30_days

要求。

查看https://stackoverflow.com/a/49866033/132438问题，询问是否在滚动窗口中专门计算唯一身份：根据需要的内存量，这是一个非常慢的操作。

当你想要滚动计数uniques时的解决方案：去寻找近似结果。

来自链接的答案：

#standardSQL
SELECT DATE_SUB(date, INTERVAL i DAY) date_grp
 , HLL_COUNT.MERGE(sketch) unique_90_day_users
 , HLL_COUNT.MERGE(DISTINCT IF(i<31,sketch,null)) unique_30_day_users
 , HLL_COUNT.MERGE(DISTINCT IF(i<8,sketch,null)) unique_7_day_users
 , COUNT(*) window_days
FROM (
  SELECT DATE(creation_date) date, HLL_COUNT.INIT(owner_user_id) sketch
  FROM `bigquery-public-data.stackoverflow.posts_questions` 
  WHERE EXTRACT(YEAR FROM creation_date)=2017
  GROUP BY 1
), UNNEST(GENERATE_ARRAY(1, 90)) i
GROUP BY 1
HAVING window_days=90
ORDER BY date_grp

Answer 2

每周计算过去30天内活跃用户数的工作解决方案。

#standardSQL
WITH days AS (
  SELECT day 
  FROM UNNEST(GENERATE_DATE_ARRAY('2018-01-01', CURRENT_DATE(), INTERVAL 1 WEEK)) AS day
), periods AS (
SELECT 
  DATE_SUB(days.day, INTERVAL 30 DAY) AS StartDate,
  days.day AS EndDate FROM days
)
SELECT
  periods.EndDate AS Day,
  COUNT(DISTINCT user_dim.device_info.resettable_device_id) as resettable_device_ids
FROM `ANDROID.app_events_*`,
  UNNEST(event_dim) AS event_dim
CROSS JOIN periods
WHERE
  PARSE_DATE("%Y%m%d", event_dim.date) BETWEEN periods.StartDate AND periods.EndDate
  AND event_dim.name = "session_start"
GROUP BY Day
ORDER BY Day DESC

移动平均最近30天

2 个答案: