有什么方法可以在雪花中使用UDF而不是进行展平?

时间:2020-05-22 09:56:17

标签: snowflake-cloud-data-platform snowflake-schema

我有下表 表格1: 有效负载(列)

{
"list": "212=1.00,214"
}

表2如下所示

table2

我希望使用UDF而不是使用flatten来获得如下所示的结果

{
  "test13": {
    "code": "212",
    "desc": "success",
    "value": "1.00"
  },
  "test15": {
    "code": "214",
    "desc": "Impression",
    "value": ""
  }
}

2 个答案:

答案 0 :(得分:0)

您应该能够执行JavaScript UDTF(用户定义的表函数)https://docs.snowflake.com/en/sql-reference/udf-js-table-functions.html,该UDTF可以采用单行有效负载并返回多行。

答案 1 :(得分:0)

因此,SQL可以做到这一点,我知道您不需要:

with table1 AS (               
  select parse_json('{"list": "212=1.00,214"}') as payload
), table2 AS (
  select parse_json(column1) as payload
    ,column2 as key
    ,column3 as value
  from values ('{"id":"212"}', 'test13', 'success' ),
              ('{"id":"214"}', 'test15', 'impression' )
), table1_demunged AS (
  select split(f.value,'=')[0] as id
    ,split(f.value,'=')[1] as val
  from table1 t, lateral flatten(input=>split(t.payload:list,',')) f
), tables_joined as (
  select t2.key as obj_key
    ,t1.id as code
    ,t2.value as desc
    ,t1.val as value
  from table2 t2
  join table1_demunged t1 on t2.payload:id = t1.id
), as_objects AS (
    select obj_key, object_construct('code', code, 'desc', desc, 'value', coalesce(value,'')) as res
    from tables_joined t
)
select object_agg(obj_key, res) object
from as_objects
group by true;

给出您想要的结果:

OBJECT
{    "test13": {      "code": "212",      "desc": "success",      "value": "1.00"    },    "test15": {      "code": "214",      "desc": "impression",      "value": ""    }  }

但是我不明白您是否真的想让UDF做到所有这些,因为它是FLATTEN然后是JOIN然后是一些OBJECT_函数,或者您只是想避免使用FALTTEN,因为它是“棘手的SQL,并且想要在UDF后面找到它”,或者您正在使用某些无法解析=>的系统,因此您需要隐藏在UDF后面的扁平化组件,但在那种情况下,UDF无法为您完成所有联接。 >

感觉好像问题比被问的更多。