如何在SQL中生成令人难以置信的大数组

时间:2019-06-03 07:48:05

标签: sql google-bigquery

在尝试使用以下命令生成大型数组时

google.api_core.exceptions.BadRequest: 400 GENERATE_ARRAY(1467331200, 1530403201, 15) produced too many elements

我遇到以下错误:

authenticator: AllowAllAuthenticator

有没有一种方法可以生成上述大小的数组?

1 个答案:

答案 0 :(得分:2)

结果元素的数量上限为1048575。

        select * from
      (
          SELECT jobs.JobID, school.School_Name, 
           CONCAT(staff.First_Name, ' ', staff.Last_Name) AS 'Full_Name', 
           staff.Phone_Number, role.Role,
        row_number()over(partition by role.Role order by jobs.JobID desc) rn
      FROM jobs
     INNER JOIN school ON jobs.SchoolID=school.SchoolID
     INNER JOIN staff ON jobs.StaffID=staff.staffID
     INNER JOIN role ON jobs.roleID=role.roleID
     ) a where a.rn=1

documentation中没有提及此限制,因此建议您在页面上发送文档反馈,或者提出功能请求以增加该限制,或者在可能的情况下删除该限制。

可能的解决方法是连接阵列。 示例:Test: bq query --dry_run --nouse_legacy_sq "[replace query below]" Query: select GENERATE_ARRAY(1, 1048575) as test_array; Output: Query successfully validated. Assuming the tables are not modified, running this query will process 0 bytes of data. Query: select GENERATE_ARRAY(1, 1048576) as test_arr; Output: GENERATE_ARRAY(1, 1048576, 1) produced too many elements