Bigquery查找条目属于

时间:2017-07-06 11:13:14

标签: google-bigquery

我正在清理我的生产数据集,要做,我正在寻找一些无用的条目。

SELECT count(pid_like) FROM TABLE_DATE_RANGE(DATASET.TABLE_PRODUCTION_, DATE_ADD(CURRENT_TIMESTAMP(), -30, 'DAY'),CURRENT_TIMESTAMP()) where c1 is null and c2 is null and c3 is null and c4 is null

所以我想删除c1,c2,c3和c4为空的条目。

在我在最后30张桌子上做清洁之前:

# done for each last 30 tables
DELETE FROM DATASET.TABLE_PRODUCTION_YYYYMMDD where c1 is null and c2 is null and c2 is null and c4 is null. 

但是在清理之后,一些无用的条目仍然存在,但我无法找到他们在哪张桌子上。

1 个答案:

答案 0 :(得分:2)

尝试此操作以查找哪些表包含空值的行:

#standardSQL
SELECT
  _TABLE_SUFFIX AS suffix,
  COUNT(*) AS null_count
FROM `DATASET.TABLE_PRODUCTION_*`
WHERE _TABLE_SUFFIX BETWEEN
    FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)) AND
    FORMAT_DATE('%Y%m%d', CURRENT_DATE()) AND
  c1 IS NULL AND
  c2 IS NULL AND
  c3 IS NULL AND
  c4 IS NULL
GROUP BY suffix
HAVING null_count > 0
ORDER BY null_count DESC;

它将返回带有空行和计数的后缀(日期)。