Question

我遇到BigQuery的大问题。我有一个日常工作，在计算引擎中的VM上运行，偶尔会生成太小而不正确的表。重新运行通常会解决问题，但这种情况越来越频繁发生，所以我正在调查它。

昨天我找到了一份完成的工作，它在VM上运行时给了我正确的结果。我在BQ前端重新运行它，它返回了大约10％的行。我重新跑了3次，每次都错了。我今天跑了，它现在正确了！

我关闭了缓存。源表从未改变过。查询非常粗糙，它处理大约50Gb并使用UDF，但它永远不会失败。

造成这些不一致的原因是什么？我可以达到一些不会导致失败的资源限制吗？

谢谢

SELECT
  _a.Date Date,
  _d.Site Site,
  _a.Product Product,
  _a.AdUnitId AdUnitId,
  _a.advertiserId AdvertiserId,
  _a.OrderId OrderId,
  _a.LineItemId LineItemId,
  _a.CreativeId CreativeId,
  _a.Country Country,
  _a.CreativeSize CreativeSize,
  _a.RequestedAdUnitSizes RequestedAdUnitSizes,
  _a.Format Format,
  _a.OOP OOP,
  _a.Mode Mode,
  REPLACE(_a.PageType,'%2c',' ') PageType,
  IF(_a.PageType CONTAINS 'gallery',TRUE,FALSE) IsGallery,
  _a.Placement Placement,
  _a.Position Position,
  _a.IsCompanion IsCompanion,
  IF(_a.DeviceCategory CONTAINS 'Mobile','Mobile', _a.DeviceCategory) DeviceCategory,
  SUM(IF(_a.LineItemId=0,1,0)) UnfilledImpressions,
  SUM(IF(NOT _a.Product='Ad Server',INTEGER(_a.RowCount),IF(NOT _a.LineItemId=0,INTEGER(_a.RowCount),0))) TotalImpressions,
  SUM(_a.ActiveViewEligibleImpressions) ActiveViewEligibleImpressions,
  SUM(_b.MeasurableImpressions) + (SUM(_a.ActiveViewEligibleImpressions) - SUM(_b.NonMeasurableImpressions) - SUM(_b.ViewableImpressions)) MeasurableImpressions,
  SUM(_b.ViewableImpressions) ViewableImpressions,
  SUM(_c.RowCount) Clicks
FROM (
  SELECT
    CONCAT( IFNULL(Date,''),IFNULL(STRING(TimeUsec),''),IFNULL(Product,''),IFNULL(STRING(AdUnitId),''),IFNULL(STRING(AdvertiserId),''),IFNULL(STRING(OrderId),''),IFNULL(STRING(LineItemId),''),IFNULL(STRING(CreativeId),''),IFNULL(Country,''),IFNULL(CreativeSize,''),IFNULL(RequestedAdUnitSizes,''),IFNULL(Format,''),IFNULL(OOP,''),IFNULL(Mode,''),IFNULL(PageType,''),IFNULL(Placement,''),IFNULL(Position,''),IFNULL(STRING(IsCompanion),''),IFNULL(DeviceCategory,'')) key,
    Date,
    Product,
    AdUnitId,
    AdvertiserId,
    OrderId,
    LineItemId,
    CreativeId,
    Country,
    CreativeSize,
    RequestedAdUnitSizes,
    Format,
    OOP,
    Mode,
    PageType,
    Placement,
    Position,
    IsCompanion,
    DeviceCategory,
    COUNT(*) RowCount,
    SUM(IF(ActiveViewEligibleImpression='Y',1,0)) ActiveViewEligibleImpressions,
    SUM(IF(NOT ActiveViewEligibleImpression='Y',1,0)) NonActiveViewEligibleImpressions
  FROM
    dfpReaderImpressions(
    SELECT
      *
    FROM
      [project-id:dfp_data_transfer.dfp_NetworkImpressions_319223_20161021],
      [project-id:dfp_data_transfer.dfp_NetworkBackfillImpressions_319223_20161021],
      [project-id:dfp_data_transfer.dfp_NetworkImpressions_319223_20161022],
      [project-id:dfp_data_transfer.dfp_NetworkBackfillImpressions_319223_20161022])
  WHERE
    REPLACE(Date,'-','')='20161021'
  GROUP EACH BY
  1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19 ) _a
LEFT JOIN (
  SELECT
    CONCAT( IFNULL(Date,''),IFNULL(STRING(TimeUsec),''),IFNULL(Product,''),IFNULL(STRING(AdUnitId),''),IFNULL(STRING(AdvertiserId),''),IFNULL(STRING(OrderId),''),IFNULL(STRING(LineItemId),''),IFNULL(STRING(CreativeId),''),IFNULL(Country,''),IFNULL(CreativeSize,''),IFNULL(RequestedAdUnitSizes,''),IFNULL(Format,''),IFNULL(OOP,''),IFNULL(Mode,''),IFNULL(PageType,''),IFNULL(Placement,''),IFNULL(Position,''),IFNULL(STRING(IsCompanion),''),IFNULL(DeviceCategory,'')) key,
    SUM(IF(MeasurableImpression='Y',1,0)) MeasurableImpressions,
    SUM(IF(NOT MeasurableImpression='Y',1,0)) NonMeasurableImpressions,
    SUM(IF(ViewableImpression='Y',1,0)) ViewableImpressions,
    SUM(IF(NOT ViewableImpression='Y',1,0)) NonViewableImpressions
  FROM
    dfpReaderActiveViews(
    SELECT
      *
    FROM
      [project-id:dfp_data_transfer.dfp_NetworkActiveViews_319223_20161021],
      [project-id:dfp_data_transfer.dfp_NetworkBackfillActiveViews_319223_20161021],
      [project-id:dfp_data_transfer.dfp_NetworkActiveViews_319223_20161022],
      [project-id:dfp_data_transfer.dfp_NetworkBackfillActiveViews_319223_20161022])
  WHERE
    REPLACE(Date,'-','')='20161021'
  GROUP EACH BY
    1 ) _b
ON
  _a.key = _b.key
LEFT JOIN (
  SELECT
    CONCAT( IFNULL(Date,''),IFNULL(STRING(TimeUsec),''),IFNULL(Product,''),IFNULL(STRING(AdUnitId),''),IFNULL(STRING(AdvertiserId),''),IFNULL(STRING(OrderId),''),IFNULL(STRING(LineItemId),''),IFNULL(STRING(CreativeId),''),IFNULL(Country,''),IFNULL(CreativeSize,''),IFNULL(RequestedAdUnitSizes,''),IFNULL(Format,''),IFNULL(OOP,''),IFNULL(Mode,''),IFNULL(PageType,''),IFNULL(Placement,''),IFNULL(Position,''),IFNULL(STRING(IsCompanion),''),IFNULL(DeviceCategory,'')) key,
    COUNT(*) RowCount
  FROM
    dfpReaderClicks(
    SELECT
      *
    FROM
      [project-id:dfp_data_transfer.dfp_NetworkClicks_319223_20161021],
      [project-id:dfp_data_transfer.dfp_NetworkBackfillClicks_319223_20161021],
      [project-id:dfp_data_transfer.dfp_NetworkClicks_319223_20161022],
      [project-id:dfp_data_transfer.dfp_NetworkBackfillClicks_319223_20161022])
  WHERE
    REPLACE(Date,'-','')='20161021'
  GROUP EACH BY
    1 ) _c
ON
  _a.key = _c.key
INNER JOIN (
  SELECT
    AdUnitId,
    Site,
    AdUnit1Name
  FROM
    [dfp_data_transfer.usr_Ad_Units_Active] ) _d
ON
  _a.AdUnitId = _d.AdUnitId
GROUP BY
  1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20

BigQuery提供不一致的结果

0 个答案: