Postgresql:创建表时UNION ALL比UNION慢?

时间:2017-04-09 07:29:37

标签: postgresql union common-table-expression

我有一个UNION查询。从EXPLAIN输出中,我注意到UNION在追加后总是进行排序。所以我把它改为UNION ALL

据说查询应该更快? 但实际执行表明,使用UNION ALL时,查询总是在几秒后返回。 (也许我的数据集不够大,差异看起来微不足道,但我对我来说仍然没有意义。)我错过了什么吗?

QUERY

WITH wcal AS (...), wlog AS (...)
SELECT * INTO temp.utlog FROM (
    SELECT * FROM wcal 
    UNION SELECT * FROM wlog
) as t2;

EXPLAIN UNION

QUERY PLAN
Unique  (cost=691313.04..737080.66 rows=3051175 width=80)
  CTE wcal
  CTE wlog
  ->  Sort  (cost=691313.04..698940.98 rows=3051175 width=80)
        Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wid
        ->  Append  (cost=0.00..91535.25 rows=3051175 width=80)
              ->  CTE Scan on wcal  (cost=0.00..52600.00 rows=2630000 width=60)
              ->  Subquery Scan on "*SELECT* 2"  (cost=0.00..12635.25 rows=421175 width=64)
                    ->  CTE Scan on wlog  (cost=0.00..8423.50 rows=421175 width=64)

EXPLAIN UNION ALL

QUERY PLAN
Append  (cost=0.00..91535.25 rows=3051175 width=80)
  CTE wcal ...
  CTE wlog ...
  ->  CTE Scan on wcal  (cost=0.00..52600.00 rows=2630000 width=60)
  ->  Subquery Scan on "*SELECT* 2"  (cost=0.00..12635.25 rows=421175 width=64)
        ->  CTE Scan on wlog  (cost=0.00..8423.50 rows=421175 width=64)

实际结果;

UNION

Table temp.utlog dropped

Execution time: 0.02s
Statement 1 of 5 finished

0 rows affected
WITH executed successfully

Execution time: 6.05s
Statement 2 of 5 finished

UNION ALL

Table temp.utlog1 dropped

Execution time: 0.02s
Statement 1 of 5 finished

0 rows affected
WITH executed successfully

Execution time: 6.81s
Statement 2 of 5 finished

结果新表中的计数相同......

SELECT count(*) FROM temp.utlog1
UNION ALL
SELECT count(*) FROM temp.utlog

count
364414
364414

EXPLAIN ANALYZE UNION ALL

QUERY PLAN
Append  (cost=0.00..11658.36 rows=388612 width=80) (actual time=0.670..3382.220 rows=364414 loops=1)
  CTE wcal ...
  CTE wlog ...
  ->  CTE Scan on wcal  (cost=0.00..720.00 rows=36000 width=60) (actual time=0.668..140.251 rows=33624 loops=1)
  ->  Subquery Scan on "*SELECT* 2"  (cost=0.00..10578.36 rows=352612 width=64) (actual time=2635.901..3179.006 rows=330790 loops=1)
        ->  CTE Scan on wlog  (cost=0.00..7052.24 rows=352612 width=64) (actual time=2635.897..3043.421 rows=330790 loops=1)
Planning time: 1.759 ms
Execution time: 6570.793 ms
...
Planning time: 1.713 ms
Execution time: 5883.275 ms
...
Planning time: 1.351 ms
Execution time: 7092.713 ms
...
Planning time: 1.318 ms
Execution time: 5803.567 ms

EXPLAIN ANALYZE UNION

QUERY PLAN
Unique  (cost=65006.04..70835.22 rows=388612 width=80) (actual time=4259.243..4534.226 rows=364414 loops=1)
  CTE wcal ...
  CTE wlog ...
  ->  Sort  (cost=65006.04..65977.57 rows=388612 width=80) (actual time=4259.239..4370.691 rows=364414 loops=1)
        Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wid
        Sort Method: external sort  Disk: 11832kB
        ->  Append  (cost=0.00..11658.36 rows=388612 width=80) (actual time=0.677..3486.529 rows=364414 loops=1)
              ->  CTE Scan on wcal  (cost=0.00..720.00 rows=36000 width=60) (actual time=0.676..120.320 rows=33624 loops=1)
              ->  Subquery Scan on "*SELECT* 2"  (cost=0.00..10578.36 rows=352612 width=64) (actual time=2760.279..3309.084 rows=330790 loops=1)
                    ->  CTE Scan on wlog  (cost=0.00..7052.24 rows=352612 width=64) (actual time=2760.275..3189.444 rows=330790 loops=1)
Planning time: 1.500 ms
Execution time: 7577.380 ms
...
Planning time: 1.375 ms
Execution time: 6777.699 ms
...
Planning time: 1.375 ms
Execution time: 6777.699 ms
...
Planning time: 1.340 ms
Execution time: 6964.954 ms

UNION(0.。)和UNION ALL(4.。)没有SELECT INTO

WITH executed successfully

Execution time: 0.41s
Statement 1 of 4 finished

WITH executed successfully

Execution time: 0.28s
Statement 1 of 4 finished

WITH executed successfully

Execution time: 0.33s
Statement 1 of 4 finished

WITH executed successfully

Execution time: 0.25s
Statement 1 of 4 finished

WITH executed successfully

Execution time: 0.28s
Statement 1 of 5 finished

WITH executed successfully

Execution time: 4.67s
Statement 1 of 5 finished

WITH executed successfully

Execution time: 4.17s
Statement 1 of 5 finished

WITH executed successfully

Execution time: 4.17s
Statement 1 of 5 finished

WITH executed successfully

Execution time: 4.33s
Statement 1 of 5 finished

UNION没有CTE

QUERY PLAN
Unique  (cost=699529.52..752925.08 rows=3051175 width=65) (actual time=3041.962..3488.246 rows=364414 loops=1)
  ->  Sort  (cost=699529.52..707157.46 rows=3051175 width=65) (actual time=3041.958..3296.505 rows=364414 loops=1)
        Sort Key: s1.s1, ((date_trunc('day'::text, s1.s1))::date), (NULL::character varying(3)), s2.wline, (NULL::boolean), (NULL::integer)
        Sort Method: external merge  Disk: 12544kB
        ->  Append  (cost=0.00..120611.73 rows=3051175 width=65) (actual time=0.729..2218.973 rows=364414 loops=1)
              ->  Nested Loop  (cost=0.00..46077.88 rows=2630000 width=36) (actual time=0.727..94.359 rows=33624 loops=1)
              ...
              ->  WindowAgg  (cost=198.08..44022.11 rows=421175 width=24) (actual time=0.737..2049.979 rows=330790 loops=1)
              ...

Planning time: 1.318 ms
Execution time: 6438.852 ms

UNION CTE

QUERY PLAN
Unique  (cost=670453.04..723848.60 rows=3051175 width=65) (actual time=3726.279..3973.870 rows=364414 loops=1)
  CTE wcal
    ->  Nested Loop  (cost=0.00..46077.88 rows=2630000 width=36) (actual time=1.788..82.337 rows=33624 loops=1)
    ...
  CTE wlog
    ->  Sort  (cost=92006.40..93059.34 rows=421175 width=24) (actual time=2542.472..2743.255 rows=330790 loops=1)
          Sort Key: tbl_line_evts.evt_time
          Sort Method: external merge  Disk: 11712kB
          ->  WindowAgg  (cost=198.08..44022.11 rows=421175 width=24) (actual time=0.932..1982.497 rows=330790 loops=1)
          ...
  ->  Sort  (cost=670453.04..678080.98 rows=3051175 width=65) (actual time=3726.275..3823.236 rows=364414 loops=1)
        Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wdirty, wcal.wid
        Sort Method: external sort  Disk: 12584kB
        ->  Append  (cost=0.00..91535.25 rows=3051175 width=65) (actual time=1.800..3082.954 rows=364414 loops=1)
              ->  CTE Scan on wcal  (cost=0.00..52600.00 rows=2630000 width=57) (actual time=1.798..125.699 rows=33624 loops=1)
              ->  CTE Scan on wlog  (cost=0.00..8423.50 rows=421175 width=49) (actual time=2542.476..2915.524 rows=330790 loops=1)
Planning time: 2.381 ms
Execution time: 5950.324 ms

UNION ALL没有CTE

QUERY PLAN
Sort  (cost=721442.46..729070.40 rows=3051175 width=80) (actual time=3416.176..3627.253 rows=364414 loops=1)
  Sort Key: s1.s1
  Sort Method: external merge  Disk: 11816kB
  ->  Append  (cost=0.00..121664.67 rows=3051175 width=80) (actual time=0.675..2726.980 rows=364414 loops=1)
        ->  Nested Loop  (cost=0.00..46077.88 rows=2630000 width=39) (actual time=0.674..118.694 rows=33624 loops=1)
        ...
        ->  Subquery Scan on "*SELECT* 2"  (cost=198.08..49286.79 rows=421175 width=51) (actual time=1.223..2517.292 rows=330790 loops=1)
              ->  WindowAgg  (cost=198.08..45075.04 rows=421175 width=51) (actual time=1.218..2334.702 rows=330790 loops=1)
                ...
Planning time: 1.360 ms
Execution time: 5795.449 ms

UNION ALL CTE

QUERY PLAN
Append  (cost=0.00..91535.25 rows=3051175 width=80) (actual time=0.718..3595.377 rows=364414 loops=1)
  CTE wcal
    ->  Nested Loop  (cost=0.00..46077.88 rows=2630000 width=39) (actual time=0.709..86.700 rows=33624 loops=1)
    ...
  CTE wlog
    ->  Sort  (cost=98820.34..99873.28 rows=421175 width=51) (actual time=2818.220..3049.432 rows=330790 loops=1)
          Sort Key: tbl_line_evts.evt_time
          Sort Method: external merge  Disk: 10952kB
          ->  WindowAgg  (cost=198.08..45075.04 rows=421175 width=51) (actual time=0.884..2224.477 rows=330790 loops=1)
            ...
  ->  CTE Scan on wcal  (cost=0.00..52600.00 rows=2630000 width=60) (actual time=0.716..129.933 rows=33624 loops=1)
  ->  Subquery Scan on "*SELECT* 2"  (cost=0.00..12635.25 rows=421175 width=64) (actual time=2818.229..3400.468 rows=330790 loops=1)
        ->  CTE Scan on wlog  (cost=0.00..8423.50 rows=421175 width=64) (actual time=2818.224..3250.670 rows=330790 loops=1)
Planning time: 2.415 ms
Execution time: 6223.041 ms

1 个答案:

答案 0 :(得分:0)

您的假设会调和docs

  

UNION ...以与结果相同的方式从结果中删除重复的行   DISTINCT,除非使用UNION ALL。

(...)我的

如果检查两个查询的成本,您会发现UNION ALL的成本要低得多。 91K vs 737K。现在为了解释与实际结果的不同,我认为这是io缓存和写入操作的结果,尝试在不创建表的情况下测量时间,而只是运行10解释使用UNION进行分析,然后进行10次使用UNION ALL。应该不那么令人惊讶。