大型查询视图已经开始破坏以前用于处理扁平化重复字段的工作

时间:2017-09-06 21:46:35

标签: google-bigquery

“我有很多观点,我已经用这个基本结构开发了

{ 
    "bar": "barvalue",
    "foo":[
        { "nested": "value" },
        { "nested":"value2"}
}

所以我有以下观点可以工作

select
  bar as bar,
  foo.nested as nested
flatten([table name],foo)

但是更复杂的观点已经在本周停止了。使用旧错误“无法保存视图。无法为字段成员创建有效的输出架构”。重命名嵌套到foo.nested我曾经用flatten解决。有人见过这个吗?有解决方案吗我的视图加载了完全相同的语法失败,所以很确定这是一个新的“功能”“bug”

所以我现在失败的特定值就像这样

    Last modified                     Schema                    Total Rows   Total Bytes   Expiration   Labels
 ----------------- ------------------------------------------ ------------ ------------- ------------ --------
  06 Sep 22:23:45   |- description: string                     4            1450
                    |- adminCreated: boolean
                    +- members: record (repeated)
                    |  |- status: string
                    |  |- kind: string
                    |  |- email: string
                    |  |- etag: string
                    |  |- role: string
                    |  |- type: string
                    |  |- id: string
                    +- scanner: record
                    |  |- scanversion: float
                    |- id: string
                    +- aliases: record (repeated)
                    |  |- alias: string
                    +- nonEditableAliases: record (repeated)
                    |  |- alias: string
                    |- kind: string
                    |- name: string
                    |- directMembersCount: string
                    |- etag: string
                    |- email: string

和另一个

    Last modified                 Schema                 Total Rows   Total Bytes   Expiration   Labels
 ----------------- ----------------------------------- ------------ ------------- ------------ --------
  06 Sep 22:23:46   |- displayName: string              11           20361
                    |- name: string
                    |- organizationId: string
                    |- creationTime: string
                    |- lifecycleState: string
                    +- owner: record
                    |  |- directoryCustomerId: string
                    +- bindings: record (repeated)
                    |  |- role: string
                    |  +- members: record (repeated)
                    |  |  |- member: string
                    +- scanner: record
                    |  |- scanversion: float

然后像这样的视图因错误而失败。

SELECT
  "groups/"+email AS name,
  g.members.email AS member,
  g.members.role AS role,
  REGEXP_EXTRACT(g.members.email,r'.*@(.*)') AS domain
FROM
  FLATTEN([<projectid>:<dataset>.group],members) g
JOIN
 [<projectid>:<dataset>.organization] o
ON
  o.scanner.scanversion = g.scanner.scanversion

1 个答案:

答案 0 :(得分:1)

好的,所以在进一步跟踪和错误后,我找到了改变视图的工作。

SELECT
  violation,
  name,
  parent,
  members.role as role,
  members.member as member
FROM (
  SELECT
    "Invalid group member not in %orgdescription% domains" AS violation,
    name,
    parent,
    role AS members.role,
    member AS members.member
  FROM (
    SELECT
      "groups/"+g.email AS name,
      o.name AS parent,
      g.members.role AS role,
      g.members.email AS member,
      REGEXP_EXTRACT(g.members.email,r'.*@(.*)') AS domain
    FROM
      FLATTEN([<projectid>:<dataset>.group],members) g
    JOIN
      [<projectid>:<dataset>.organizationhd] o
    ON
      o.scanner.scanversion = g.scanner.scanversion
    WHERE
      g.members.email IS NOT NULL)
  WHERE
    domain NOT IN('foo.com'))