变型表的横向扁平雪花

时间:2020-03-05 02:29:54

标签: snowflake-cloud-data-platform flatten snowflake-schema lateral

我有一个名为raw_json的变量表,其中包含多个json文件,这些文件对于ID而言是唯一的,但每个json中都包含相似的数据点。它们位于jsontext列中。这是上下文的SS。我正在尝试将raw_json表的每一行展平为普通表视图。这两个数组索引需要对齐才能分配正确的值。

table name is raw_json

这是raw_json表中的两行,以及json的结构。

{
 "ID": "PO-103",
 "content": {
   "EEList": [
     {
       "EEListID": "PO-103-1",
       "EEProductID": "XXX1976",

     },
     {
       "EEListID": "PO-103-2",
       "EEProductID": "XXX1977",
     },
     {
       "EEListID": "PO-103-3",
       "EEProductID": "XXX1978",
     }
   ],
   "EENotesList": [
     {
       "FirstName": "John",
       "LastName": "Smith",
       "pxObjClass": "XX-XXSales-Work-XX"
     },
     {
       "FirstName": "Bob",
       "LastName": "Joe",
       "pxObjClass": "XX-XXSales-Work-XX"
     },
     {
       "FirstName": "Mike",
       "LastName": "Smith",
       "pxObjClass": "XX-XXSales-Work-XX"
     }
   ],
 }
}
{
  "ID": "PO-104",
  "content": {
    "EEList": [
      {
        "EEListID": "PO-104-1",
        "EEProductID": "XXX1979",

      },
      {
        "EEListID": "PO-104-2",
        "EEProductID": "XXX1980",
      },
      {
        "EEListID": "PO-104-3",
        "EEProductID": "XXX1981",
      }
    ],
    "EENotesList": [
      {
        "FirstName": "Sarah",
        "LastName": "Butler",
        "pxObjClass": "XX-XXSales-Work-XX"
      },
      {
        "FirstName": "Jessica",
        "LastName": "Adams",
        "pxObjClass": "XX-XXSales-Work-XX"
      }
    ],
  }
}

进入这样的表格(我需要这个)

+--------+----------+-------------+-----------+----------+-------------------+
|   ID   | EEListID | EEProductID | FirstName | LastName |    pxObjClass     |
+--------+----------+-------------+-----------+----------+-------------------+
| PO-103 | PO-103-1 | XXX1976     | John      | Smith    | X-XXSales-Work-XX |
| PO-103 | PO-103-2 | XXX1977     | Bob       | Joe      | X-XXSales-Work-XX |
| PO-103 | PO-103-3 | XXX1978     | Mike      | Smith    | X-XXSales-Work-XX |
| PO-104 | PO-104-1 | XXX1979     | Sarah     | Butler   | X-XXSales-Work-XX |
| PO-104 | PO-104-2 | XXX1980     | Jessica   | Adams    | X-XXSales-Work-XX |
+--------+----------+-------------+-----------+----------+-------------------+

我已经能够将EENoteList数组展平为一个表并为该表分配正确的ID。到目前为止,这是我的代码:我在这里出错了。在不分散表的情况下添加EEList值。

select
    jsontext:ID::varchar as ID,
    en.value:FirstName::varchar as FirstName,
    en.value:LastName::varchar as LastName,
    en.value:pxObjClass::varchar as pxObjClass
   -- concat(ID, EEProductID, FirstName, LastName)

    from raw_json,
    lateral flatten (input => jsontext:content:EENotesList, outer => false) en;

产生这张表的(我有这个)

+--------+-----------+----------+-------------------+
|   ID   | FirstName | LastName |    pxObjClass     |
+--------+-----------+----------+-------------------+
| PO-103 | John      | Smith    | X-XXSales-Work-XX |
| PO-103 | Bob       | Joe      | X-XXSales-Work-XX |
| PO-103 | Mike      | Smith    | X-XXSales-Work-XX |
| PO-104 | Sarah     | Butler   | X-XXSales-Work-XX |
| PO-104 | Jessica   | Adams    | X-XXSales-Work-XX |
| PO-104 | Terrence  | Williams | X-XXSales-Work-XX |
+--------+-----------+----------+-------------------+

1 个答案:

答案 0 :(得分:1)

因此,大多数情况下,此答案是将数据放入CTE的序言,但是“ iff”两个数组的顺序处于同步状态,您可以仅使用flatten的索引来访问其他类型的原始数组:

WITH raw_json AS (
select PARSE_json(column1) AS jsontext FROM VALUES 
 ('{
 "ID": "PO-103",
 "content": {
   "EEList": [
     {
       "EEListID": "PO-103-1",
       "EEProductID": "XXX1976",

     },
     {
       "EEListID": "PO-103-2",
       "EEProductID": "XXX1977",
     },
     {
       "EEListID": "PO-103-3",
       "EEProductID": "XXX1978",
     }
   ],
   "EENotesList": [
     {
       "FirstName": "John",
       "LastName": "Smith",
       "pxObjClass": "XX-XXSales-Work-XX"
     },
     {
       "FirstName": "Bob",
       "LastName": "Joe",
       "pxObjClass": "XX-XXSales-Work-XX"
     },
     {
       "FirstName": "Mike",
       "LastName": "Smith",
       "pxObjClass": "XX-XXSales-Work-XX"
     }
   ],
 }
}'), ('{
  "ID": "PO-104",
  "content": {
    "EEList": [
      {
        "EEListID": "PO-104-1",
        "EEProductID": "XXX1979",

      },
      {
        "EEListID": "PO-104-2",
        "EEProductID": "XXX1980",
      },
      {
        "EEListID": "PO-104-3",
        "EEProductID": "XXX1981",
      }
    ],
    "EENotesList": [
      {
        "FirstName": "Sarah",
        "LastName": "Butler",
        "pxObjClass": "XX-XXSales-Work-XX"
      },
      {
        "FirstName": "Jessica",
        "LastName": "Adams",
        "pxObjClass": "XX-XXSales-Work-XX"
      }
    ],
  }
}') 
)
select
    jsontext:ID::varchar as ID,
    en.value:FirstName::varchar as FirstName,
    en.value:LastName::varchar as LastName,
    en.value:pxObjClass::varchar as pxObjClass,
    jsontext:content.EEList[en.index].EEListID::text as EEListID,
    jsontext:content.EEList[en.index].EEProductID::text as EEProductID
    from raw_json,
    lateral flatten (input => jsontext:content:EENotesList, outer => false) en;

这导致:

ID  FIRSTNAME   LASTNAME    PXOBJCLASS  EELISTID    EEPRODUCTID
PO-103  John    Smith   XX-XXSales-Work-XX  PO-103-1    XXX1976
PO-103  Bob     Joe     XX-XXSales-Work-XX  PO-103-2    XXX1977
PO-103  Mike    Smith   XX-XXSales-Work-XX  PO-103-3    XXX1978
PO-104  Sarah   Butler  XX-XXSales-Work-XX  PO-104-1    XXX1979
PO-104  Jessica Adams   XX-XXSales-Work-XX  PO-104-2    XXX1980