SQL嵌套数据运行总计直到达到值

时间:2017-11-06 23:12:34

标签: sql google-bigquery chrome-ux-report

我正在尝试使用这里的教程在Google BigQuery上使用复杂的嵌套数据集: https://developers.google.com/web/tools/chrome-user-experience-report/getting-started

以json格式导出的数据如下所示: https://gist.github.com/kmturley/c46eb3898d6ee62871f4402a4d2c6f7d

我正在使用的例子是: https://bigquery.cloud.google.com/savedquery/226352634162:c5b7ee9ea0394728a821cf60f58737c2

#standardSQL
SELECT
  effective_connection_type.name AS ect,
  SUM((
    SELECT
      SUM(bin.density)
    FROM
      UNNEST(first_contentful_paint.histogram.bin) bin
    WHERE
      bin.end <= 1000
      AND origin = 'http://example.com')) AS density
FROM
  `chrome-ux-report.chrome_ux_report.201710`
GROUP BY
  ect
ORDER BY
  density DESC

输出数据:

1   4G  0.6977   
2   3G  0.056    
3   slow-2G null     
4   2G  null     
5   offline null

但是当SUM(bin.density)达到0.7时,我想输出bin.end值(时间)。我的预期输出看起来像这样:

1   4G  1000     
2   3G  50000    
3   slow-2G null     
4   2G  null     
5   offline null

4G连接的含义,70%(0.7)的页面加载小于1.5秒(1500)。我试图将脚本修改为:

SELECT
  SUM(bin.density)
WHERE
  SUM(bin.density) <= 0.7

但是这是不允许的,所以尝试了:

SELECT
  SUM(bin.density) AS RunningTotal
WHERE
  RunningTotal <= 0.7

还试过

SELECT
  SUM(bin.density) OVER() AS RunningTotal
WHERE
  RunningTotal <= 0.7

但这也不起作用!如何使用嵌套数据集实现运行总计?并让它输出bin.end时间?

如果我无法使用嵌套数据集来处理SQL运行总计,那么我唯一的另一个选择是展平数据集,并使用Python循环遍历每一行以计算结果。哪个性能要低得多!

更新:根据Felipe Hoffa的回答解决方案

#standardSQL
SELECT origin, form, ect, `end`, density
FROM (
  SELECT origin, form, ect, `end`, density, ROW_NUMBER() OVER(PARTITION BY ect ORDER BY `end` DESC) rn
  FROM (
    SELECT origin, form, ect, bin.end, SUM(bin.density) OVER(PARTITION BY ect ORDER BY `end`) AS density 
    FROM (
      SELECT origin, form_factor.name form, effective_connection_type.name ect, first_contentful_paint.histogram
      FROM    `chrome-ux-report.chrome_ux_report.201710`
      WHERE    origin = 'http://example.com' AND form_factor.name = 'phone'
    ) , UNNEST(histogram.bin) AS bin
  )
  WHERE density < 0.7
) 
WHERE rn=1

https://bigquery.cloud.google.com/savedquery/88263730615:ba1906e86b074511a804660ec973de37

2 个答案:

答案 0 :(得分:2)

使用累积SUM()并按连接排序结果:

#standardSQL
SELECT ect, `end`, density
FROM (
  SELECT ect, `end`, density, ROW_NUMBER() OVER(PARTITION BY ect ORDER BY `end` DESC) rn
  FROM (
    SELECT  ect, bin.end, SUM(bin.density) OVER(PARTITION BY ect ORDER BY `end`) AS density 
    FROM (
      SELECT  effective_connection_type.name ect, first_contentful_paint.histogram
      FROM    `chrome-ux-report.chrome_ux_report.201710`
      WHERE    origin = 'http://example.com'
    ) , UNNEST(histogram.bin) AS bin
  )
  WHERE density < 0.7
) 
WHERE rn=1

enter image description here

答案 1 :(得分:0)

我想你想要一个子查询和累积总和:

with cte as (
      <your query here>
     )
select x.*
from (select cte.*, sum(density) over (order by density desc) as runningtotal
      from cte
     ) x
where runningtotal - density < 0.7 and runningtotal >= 0.7;