如何优化SQL查询以检查表中列值的一致性

时间:2015-12-24 11:42:35

标签: postgresql amazon-redshift

我想在多个表中检查每个表中是否存在相同的键/相同数量的键。

目前我已经创建了一个解决方案,用于检查每个表的密钥数,在所有表合并在一起时检查密钥数,然后进行比较。

此解决方案有效,但我想知道是否有更优化的解决方案...

目前的示例解决方案:

SELECT COUNT(DISTINCT variable) AS num_ids FROM table_a;

SELECT COUNT(DISTINCT variable) AS num_ids FROM table_b;

SELECT COUNT(DISTINCT variable) AS num_ids FROM table_c;

SELECT COUNT(DISTINCT a.variable) AS num_ids
FROM (SELECT DISTINCT VARIABLE FROM table_a) a
  INNER JOIN (SELECT DISTINCT VARIABLE FROM table_b) b ON a.variable = b.variable
  INNER JOIN (SELECT DISTINCT VARIABLE FROM table_c) c ON a.variable = c.variable;

更新:

我在一个查询中将这一点放在一起很困难的是,任何表格在我想查看的VARIABLE上可能都不是唯一的,所以我必须在合并之前使用distinct避免扩大连接

2 个答案:

答案 0 :(得分:2)

由于我们只是在计算,我认为没有必要加入variable列的表格。 UNION应该足够了。 我们仍然必须使用DISTINCT来忽略/禁止重复,这通常意味着额外的排序。 variable上的索引应该有助于获取单独表的计数,但它对获取组合表的计数没有帮助。

以下是比较两个表的示例:

WITH
CTE_A
AS
(
    SELECT COUNT(DISTINCT variable) AS CountA
    FROM TableA
)
,CTE_B
AS
(
    SELECT COUNT(DISTINCT variable) AS CountB
    FROM TableB
)
,CTE_AB
AS
(
    SELECT COUNT(DISTINCT variable) AS CountAB
    FROM
    (
        SELECT variable
        FROM TableA

        UNION ALL 
        -- sic! use ALL here to avoid sort when merging two tables
        -- there should be only one distinct sort for the outer `COUNT`

        SELECT variable
        FROM TableB
    ) AS AB
)
SELECT
    CASE WHEN CountA = CountAB AND CountB = CountAB 
    THEN 'same' ELSE 'different' END AS ResultAB
FROM
    CTE_A
    CROSS JOIN CTE_B
    CROSS JOIN CTE_AB
;

三张桌子:

WITH
CTE_A
AS
(
    SELECT COUNT(DISTINCT variable) AS CountA
    FROM TableA
)
,CTE_B
AS
(
    SELECT COUNT(DISTINCT variable) AS CountB
    FROM TableB
)
,CTE_C
AS
(
    SELECT COUNT(DISTINCT variable) AS CountC
    FROM TableC
)
,CTE_ABC
AS
(
    SELECT COUNT(DISTINCT variable) AS CountABC
    FROM
    (
        SELECT variable
        FROM TableA

        UNION ALL 
        -- sic! use ALL here to avoid sort when merging two tables
        -- there should be only one distinct sort for the outer `COUNT`

        SELECT variable
        FROM TableB

        UNION ALL 
        -- sic! use ALL here to avoid sort when merging two tables
        -- there should be only one distinct sort for the outer `COUNT`

        SELECT variable
        FROM TableC
    ) AS AB
)
SELECT
    CASE WHEN CountA = CountABC AND CountB = CountABC AND CountC = CountABC 
    THEN 'same' ELSE 'different' END AS ResultABC
FROM
    CTE_A
    CROSS JOIN CTE_B
    CROSS JOIN CTE_C
    CROSS JOIN CTE_ABC
;

我故意选择CTE,因为据我所知Postgres实现了CTE,在我们的例子中,每个CTE只有一行。

如果在redshift上可用,则使用array_agg 排序是更好的变体。您仍然需要使用DISTINCT,但您不必将所有表合并在一起。

WITH
CTE_A
AS
(
    SELECT array_agg(DISTINCT variable ORDER BY variable) AS A
    FROM TableA
)
,CTE_B
AS
(
    SELECT array_agg(DISTINCT variable ORDER BY variable) AS B
    FROM TableB
)
,CTE_C
AS
(
    SELECT array_agg(DISTINCT variable ORDER BY variable) AS C
    FROM TableC
)
SELECT
    CASE WHEN A = B AND B = C
    THEN 'same' ELSE 'different' END AS ResultABC
FROM
    CTE_A
    CROSS JOIN CTE_B
    CROSS JOIN CTE_C
;

答案 1 :(得分:0)

好吧,这可能是我可以为你构建的最糟糕的SQL :)我将永远否认我写了这个并且我的stackoverflow帐户被黑了;)

SELECT
  'All OK'
WHERE
  ( SELECT COUNT(DISTINCT id) FROM table_a ) = ( SELECT COUNT(DISTINCT id) FROM table_b )
  AND ( SELECT COUNT(DISTINCT id) FROM table_b ) = ( SELECT COUNT(DISTINCT id) FROM table_c )

顺便说一句,这不会优化查询 - 它仍然会进行三次查询(但我猜它比4更好?)。

更新:根据您的用例如下:新的sql小提琴http://sqlfiddle.com/#!15/a0403/1

SELECT DISTINCT
  tbl_a.a_count,
  tbl_b.b_count,
  tbl_c.c_count
FROM
  ( SELECT COUNT(id) a_count, array_agg(id order by id) ids FROM table_a) tbl_a,
  ( SELECT COUNT(id) b_count, array_agg(id order by id) ids FROM table_b) tbl_b,
  ( SELECT COUNT(id) c_count, array_agg(id order by id) ids FROM table_c) tbl_c
WHERE
  tbl_a.ids = tbl_b.ids
  AND tbl_b.ids = tbl_c.ids

只有当所有表具有相同的行数时,上述查询才会返回,从而确保IDS也相同。