如何创建结构数组的火花SQL

时间:2020-10-09 23:29:57

标签: sql apache-spark apache-spark-sql

我正在尝试创建array<struct>类型。我的代码似乎返回了我想要的,但是当我打开json文件时,数组仅包含1个结构。

select idCol,
collect_list(
named_struct("name", "apple", "amount", named_struct("value", apple_amount, "currency", currency),
             "name", "orange", "amount", named_struct("value", orange_amount, "currency", currency),
             "name", "banana", "amount", named_struct("value", banana_amount, "currency", currency))) as categories
  from (
     select
       1 as idCol,
       'usd' as currency,
       20 as apple_amount,
       30 as orange_amount,
       40 as banana_amount)
  group by 1

返回

{
  "idCol": 1,
  "categories": [
    {
      "name": "apple",
      "amount": {
        "value": 20,
        "currency": "usd"
      },
      "name": "orange",
      "amount": {
        "value": 30,
        "currency": "usd"
      },
      "name": "banana",
      "amount": {
        "value": 40,
        "currency": "usd"
      }
    }
  ]
}

我如何使其成为这个?

{
      "idCol": 1,
      "categories": [
        {
          "name": "apple",
          "amount": {
            "value": 20,
            "currency": "usd"
          }
        },
        {
          "name": "orange",
          "amount": {
            "value": 30,
            "currency": "usd"
          }
        },
        {
          "name": "banana",
          "amount": {
            "value": 40,
            "currency": "usd"
          }
        }
      ]
    }

1 个答案:

答案 0 :(得分:0)

我应该有一个狭窄的查询。

select idCol,
collect_set(named_struct("name", type, "amount", named_struct("value", amount, "currency", currency))) as categories
from (
select
 1 as idCol,
 'usd' as currency,
 'apple' as type,
 20 as amount
 union all
 select 1 as idCol,
 'usd' as currency,
 'orange' as type,
 30 as amount
 union all
 select 1 as idCol,
 'usd' as currency,
 'banana' as type,
 40 as amount)
group by 1
相关问题