如何在bigquery中处理动态模式

时间:2017-07-18 22:46:46

标签: google-bigquery

我的数据如下: 第1行 - {"id": "1", "object": "user","metadata": {"name": "1234"}} 第2行 - {"id": "1", "object": "user","metadata": {"name": "1234","email": "abc@abc.com"}}

我使用第1行创建了表格 metadata RECORD NULLABLE metadata.tenant STRING NULLABLE object STRING NULLABLE id STRING NULLABLE

但我的插入将在第2行失败。我的架构应该是什么样的,以便它可以处理元数据字段中的更改?

1 个答案:

答案 0 :(得分:2)

对于您问题中显示的示例 - 我将使用以下架构

  
[
  {
    "name": "id",
    "type": "INTEGER",
    "mode": "NULLABLE"
  },
  {
    "name": "object",
    "type": "STRING",
    "mode": "NULLABLE"
  },
  {
    "name": "metadata",
    "type": "STRING",
    "mode": "NULLABLE"
  }
]   

下面是我如何处理它的一个例子

#standardSQL
WITH `yourProject.yourDataset.yourTable` AS (
  SELECT 1 AS id, 'user' AS object, '{"name": "BI Architect", "email": "abc@abc.com"}' AS metadata UNION ALL
  SELECT 2, 'expert', '{"name": "Elliott Brossard"}'
)
SELECT 
  id, 
  object, 
  JSON_EXTRACT_SCALAR(metadata, '$.name') AS name,
  JSON_EXTRACT_SCALAR(metadata, '$.email') AS email 
FROM `yourProject.yourDataset.yourTable` 
ORDER BY id

导致输出低于

id  object  name                email    
1   user    BI Architect        abc@abc.com  
2   expert  Elliott Brossard    null