我在Google BigQuery中有如下数据:
sample_date_time_UTC time_zone milliseconds_between_samples
-------- --------- ----------------------------
2019-03-31 01:06:03 UTC Europe/Paris 60000
2019-03-31 01:16:03 UTC Europe/Paris 60000
...
期望数据采样按固定的时间间隔进行,由milliseconds_between_samples
字段的值指示:
time_zone
是代表Google Cloud Supported Timezone Value的字符串
然后,我要检查任意一天范围(对于给定的time_zone
,以本地日期表示)在任何特定日期的实际样本数量与预期数量之比:>
with data as
(
select
-- convert sample_date_time_UTC to equivalent local datetime for the timezone
DATETIME(sample_date_time_UTC,time_zone) as localised_sample_date_time,
milliseconds_between_samples
from `mytable`
where sample_date_time between '2019-03-31 00:00:00.000000+01:00' and '2019-04-01 00:00:00.000000+02:00'
)
select date(localised_sample_date_time) as localised_date, count(*)/(86400000/avg(milliseconds_between_samples)) as ratio_of_daily_sample_count_to_expected
from data
group by localised_date
order by localised_date
问题是这有一个错误,因为我已经将一天的预期毫秒数硬编码为86400000
。这是不正确的,因为夏时制从指定的time_zone
(Europe/Paris
)开始时,一天缩短了1小时。夏令时结束后,一天会延长1小时。
因此,以上查询不正确。它查询Europe/Paris
时区(即该时区开始夏令时)今年3月31日的数据。当天的毫秒数应为82800000
。
在查询中,如何获取指定的localised_date
的正确毫秒数?
更新:
我尝试这样做以查看返回的结果:
select DATETIME_DIFF(DATETIME('2019-04-01 00:00:00.000000+02:00', 'Europe/Paris'), DATETIME('2019-03-31 00:00:00.000000+01:00', 'Europe/Paris'), MILLISECOND)
那没有用-我得到86400000
答案 0 :(得分:1)
通过删除+01:00
和+02:00
,可以得到两个时间戳的毫秒数差。请注意,这给出了UTC 90000000
中的时间戳之间的时差,它与过去的实际毫秒数不同。
您可以执行以下操作以获取一天的毫秒数:
select 86400000 + (86400000 - DATETIME_DIFF(DATETIME('2019-04-01 00:00:00.000000', 'Europe/Paris'), DATETIME('2019-03-31 00:00:00.000000', 'Europe/Paris'), MILLISECOND))
答案 1 :(得分:1)
感谢@Juta,提供了使用UTC时间进行计算的提示。当我按照本地化日期对每天的数据进行分组时,我发现可以使用以下逻辑获取“本地化”日期的开始和结束日期时间(以UTC为单位)来计算每天的毫秒数:
-- get UTC start datetime for localised date
-- get UTC end datetime for localised date
-- this then gives the milliseconds for that localised date:
datetime_diff(utc_end_datetime, utc_start_datetime, MILLISECOND);
因此,我的完整查询变为:
with daily_sample_count as (
with data as
(
select
-- get the date in the local timezone, for sample_date_time_UTC
DATE(sample_date_time_UTC,time_zone) as localised_date,
milliseconds_between_samples
from `mytable`
where sample_date_time between '2019-03-31 00:00:00.000000+01:00' and '2019-04-01 00:00:00.000000+02:00'
)
select
localised_date,
count(*) as daily_record_count,
avg(milliseconds_between_samples) as daily_avg_millis_between_samples,
datetime(timestamp(localised_date, time_zone)) as utc_start_datetime,
datetime(timestamp(date_add(localised_date, interval 1 day), time_zone)) as utc_end_datetime
from data
)
select
localised_date,
-- apply calculation for ratio_of_daily_sample_count_to_expected
-- based on the actual vs expected number of samples for the day
-- no. of milliseconds in the day changes, when transitioning in/out of daylight saving - so we calculate milliseconds in the day
daily_record_count/(datetime_diff(utc_end_datetime, utc_start_datetime, MILLISECOND)/daily_avg_millis_between_samples) as ratio_of_daily_sample_count_to_expected
from
daily_sample_count