我正在将spark sql迁移到snowsql。 有一次我遇到了一个场景,其中我在spark sql中使用了嵌套窗口函数。而且我想将该SQL查询迁移到雪花中。但是雪花不支持嵌套窗口功能。
火花SQL查询-
SELECT
*,
(case when (
(
lead(timestamp -lag(timestamp)
over (partition by session_id order by timestamp))
over (partition by session_id order by timestamp)
) is not null)
then
(
lead(timestamp -lag(timestamp)
over (partition by session_id order by timestamp))
over (partition by session_id order by timestamp)
)
else 0 end)/1000 as pg_to_pg
FROM dwell_time_step2
已转换的Snowsql-
with lagsession as (
SELECT
a.*,
lag(timestamp) over (partition BY session_id order by timestamp asc) lagsession
FROM mktg_web_wi.dwell_time_step2 a
)
select
a.,
nvl(lead(a.timestamp - b.lagsession) over (partition BY a.session_id order by a.timestamp),0)/1000 pg_to_pg
FROM mktg_web_wi.dwell_time_step2 a,
lagsession b
WHERE a.key=b.key
order by timestamp;
输出-
在这里,问题出在Snow-sql输出中。停留时间值已分配给不同的网址。
期望是使spark-sql查询在snowsql上工作,并且两种情况下的输出应该相同。
请让我知道是否有人知道如何解决此问题。
谢谢!!
答案 0 :(得分:1)
我认为将其从嵌套窗口函数更改为cte已经改变了记录滞后和超前所指的内容,但这很难理解。
无论如何,如果我在这里理解您的代码,我认为有一个更简单的方法,只有一个Windows函数。
select
a.*,
(nvl(lead(a.timestamp) over (partition BY a.session_id order by a.timestamp) - a.timestamp)/1000,0) pg_to_pg
FROM mktg_web_wi.dwell_time_step2 a
order by timestamp;