Question

我正在使用谷歌应用程序脚本自动执行从BigQuery导入电子表格的过程。但应用程序脚本正在回复我

Error 413: Message: response too large

这是我的代码示例，错误在第5行

1 var projectId = projectid;
2 var request = {
3 query: 'My Query'
4 };
5 var queryResults = BigQuery.Jobs.query(request, projectId);

有哪些可能的解决方案？是否有任何解决方案不会增加我的BigQuery费用？

PS：结果大约是16 MB的数据。即大约300,000行。

查询：

SELECT
  ORDER.addedon AS date,
  ORDER.display_order_id AS order_id,
  OrderSkuDetails.pid AS pid,
  OrderSkuDetails.price AS price,
  OrderSkuDetails.saleprice AS saleprice,
  OrderSkuDetails.subtotal AS subtotal,
  OrderSkuDetails.shippingcharge AS shippingcharge,
  OrderSkuDetails.codcharge AS codcharge,
  User.email AS email,
  ORDER.order_id AS payment_id,
  ORDER.payment_mode AS payment_mode,
  ORDER.source AS source,
  ORDER.user_id AS user_id,
  Payments.payment_status AS payment_status,
  User.profileJson.text,
  OrderStatus.sub_status_id AS sub_status_id,
  NProduct.featured AS featured
FROM
  FLATTEN([Mixpanel_Import.Order],payment_mode) AS ORDER
INNER JOIN
  [Mixpanel_Import.OrderSkuDetails] AS OrderSkuDetails
ON
  ORDER.order_id=OrderSkuDetails.order_id
INNER JOIN
  [Mixpanel_Import.OrderStatus] AS OrderStatus
ON
  ORDER.order_id=OrderStatus.order_id
INNER JOIN
  [Mixpanel_Import.User] AS User
ON
  ORDER.user_id=User.__key__.id
INNER JOIN
  [Mixpanel_Import.Payments] AS Payments
ON
  ORDER.order_id=Payments.order_id
INNER JOIN
  [Mixpanel_Import.NProduct] AS NProduct
ON 
  OrderSkuDetails.pid=NProduct.pid

这是代码

function bigQuery(tableName) {
  var projectId = 'project';
  var request = {
    query: The Query
  };
  var queryResults = BigQuery.Jobs.query(request, projectId); //Error on this line
  var jobId = queryResults.jobReference.jobId;
  Logger.log(jobId);
  var sleepTimeMs = 500;
  while (!queryResults.jobComplete) {
    Utilities.sleep(sleepTimeMs);
    sleepTimeMs *= 2;
    queryResults = BigQuery.Jobs.getQueryResults(projectId, jobId);
  }
  while (queryResults.pageToken) {
    queryResults.concat(BigQuery.Jobs.getQueryResults(projectId, jobId, {pageToken: queryResults.pageToken}));
  }
  return queryResults;
}

Answer 1

请尝试在作业配置中将allowLargeResults的值设置为true。

如Returning large query results

中所述

返回大结果的查询需要更长的时间才能执行，即使结果集很小，也会受到其他限制：


您必须指定目的地表。

您无法指定顶级ORDER BY，TOP或LIMIT子句。这样做会否定使用allowLargeResults的好处，因为无法再并行计算查询输出。

只有与PARTITION BY子句一起使用时，窗口函数才能返回大型查询结果。

如果这不起作用，建议的解决方案和想法可能会解决下面给出的问题也可能有所帮助：

除此之外，还可以在Troubleshooting Errors中找到有关使用BigQuery时出现的错误的信息。

应用脚本：响应对于bigquery来说太大了

1 个答案: