与原始数据库相比,在还原的转储的物化视图上搜索查询速度较慢

时间:2019-01-17 11:29:26

标签: postgresql performance restore dump materialized-views

我有一个包含各种实例化视图的psql数据库,在运行查询时,即query_a,我们在2800毫秒内完成了查询执行,然后再次重新运行同一查询,得到了53毫秒的执行时间。这可以通过psql进行缓存来解释。现在到了棘手的部分,我创建了这个数据库的转储并将其还原到NewDB中,当我重新运行query_a时,我得到了2253ms的执行时间,并且在重新运行时得到了相同的时间,即psql缓存似乎在NewDB上不起作用。 我进行了各种实验来纠正这些问题,并注意到当我明确刷新视图时并没有任何改善,但是如果我放下这些视图并在NewDB中重新创建它们,它将为我带来原始的性能。 请注意,数据在DB和NewDB中是恒定的,我已使用提到的here命令进行转储创建和还原。 在数据库上重新运行查询的结果是-> enter image description here

第一次和第二次在NewDB上运行相同查询的结果如下->

                                                                         QUERY PLAN                                                                         
------------------------------------------------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=113790614477.61..113790614477.62 rows=1 width=8) (actual time=2284.605..2284.605 rows=1 loops=1)
   Buffers: shared hit=3540872
   CTE t
     ->  Merge Join  (cost=40600.92..11846650.56 rows=763041594 width=425) (actual time=3.693..1909.916 rows=6005 loops=1)
           Merge Cond: (n.node_id = nd.node_id)
           Buffers: shared hit=3524063
           ->  Index Scan using nodes_node_id on nodes n  (cost=0.43..350865.91 rows=3824099 width=389) (actual time=0.014..1651.025 rows=3598491 loops=1)
                 Buffers: shared hit=3523372
           ->  Sort  (cost=40600.49..40700.26 rows=39907 width=40) (actual time=3.668..4.227 rows=6005 loops=1)
                 Sort Key: nd.node_id
                 Sort Method: quicksort  Memory: 623kB
                 Buffers: shared hit=691
                 ->  Bitmap Heap Scan on nodes_depths nd  (cost=1153.11..37550.73 rows=39907 width=40) (actual time=0.627..2.846 rows=6005 loops=1)
                       Recheck Cond: ((ancestor_1 = 1) OR (ancestor_2 = 1))
                       Heap Blocks: exact=658
                       Buffers: shared hit=691
                       ->  BitmapOr  (cost=1153.11..1153.11 rows=40007 width=0) (actual time=0.547..0.547 rows=0 loops=1)
                             Buffers: shared hit=33
                             ->  Bitmap Index Scan on nodes_depths_1  (cost=0.00..566.58 rows=20003 width=0) (actual time=0.032..0.032 rows=156 loops=1)
                                   Index Cond: (ancestor_1 = 1)
                                   Buffers: shared hit=4
                             ->  Bitmap Index Scan on nodes_depths_2  (cost=0.00..566.58 rows=20003 width=0) (actual time=0.515..0.515 rows=5849 loops=1)
                                   Index Cond: (ancestor_2 = 1)
                                   Buffers: shared hit=29
   ->  Merge Right Join  (cost=169565733.26..97549168801.28 rows=6491839610305 width=0) (actual time=1915.721..2284.175 rows=6005 loops=1)
         Merge Cond: (nodes_fts.node_id = t.node_id)
         Buffers: shared hit=3540872
         ->  Index Only Scan using nodes_fts_idx on nodes_fts  (cost=0.43..97055.96 rows=1701569 width=4) (actual time=0.041..277.890 rows=1598712 loops=1)
               Heap Fetches: 1598712
               Buffers: shared hit=16805
         ->  Materialize  (cost=169565732.84..173380940.81 rows=763041594 width=4) (actual time=1915.675..1916.583 rows=6005 loops=1)
               Buffers: shared hit=3524067
               ->  Sort  (cost=169565732.84..171473336.82 rows=763041594 width=4) (actual time=1915.672..1916.057 rows=6005 loops=1)
                     Sort Key: t.node_id
                     Sort Method: quicksort  Memory: 474kB
                     Buffers: shared hit=3524067
                     ->  CTE Scan on t  (cost=0.00..15260831.88 rows=763041594 width=4) (actual time=3.698..1914.771 rows=6005 loops=1)
                           Buffers: shared hit=3524063
 Planning time: 68.064 ms
 Execution time: 2285.084 ms
(40 rows)

第二次运行->

                                                                             QUERY PLAN                                                                         
------------------------------------------------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=113790614477.61..113790614477.62 rows=1 width=8) (actual time=2295.319..2295.319 rows=1 loops=1)
   Buffers: shared hit=3540868
   CTE t
     ->  Merge Join  (cost=40600.92..11846650.56 rows=763041594 width=425) (actual time=15.324..1926.744 rows=6005 loops=1)
           Merge Cond: (n.node_id = nd.node_id)
           Buffers: shared hit=3524063
           ->  Index Scan using nodes_node_id on nodes n  (cost=0.43..350865.91 rows=3824099 width=389) (actual time=0.027..1648.277 rows=3598491 loops=1)
                 Buffers: shared hit=3523372
           ->  Sort  (cost=40600.49..40700.26 rows=39907 width=40) (actual time=15.254..15.903 rows=6005 loops=1)
                 Sort Key: nd.node_id
                 Sort Method: quicksort  Memory: 623kB
                 Buffers: shared hit=691
                 ->  Bitmap Heap Scan on nodes_depths nd  (cost=1153.11..37550.73 rows=39907 width=40) (actual time=3.076..10.752 rows=6005 loops=1)
                       Recheck Cond: ((ancestor_1 = 1) OR (ancestor_2 = 1))
                       Heap Blocks: exact=658
                       Buffers: shared hit=691
                       ->  BitmapOr  (cost=1153.11..1153.11 rows=40007 width=0) (actual time=2.524..2.525 rows=0 loops=1)
                             Buffers: shared hit=33
                             ->  Bitmap Index Scan on nodes_depths_1  (cost=0.00..566.58 rows=20003 width=0) (actual time=0.088..0.088 rows=156 loops=1)
                                   Index Cond: (ancestor_1 = 1)
                                   Buffers: shared hit=4
                             ->  Bitmap Index Scan on nodes_depths_2  (cost=0.00..566.58 rows=20003 width=0) (actual time=2.434..2.435 rows=5849 loops=1)
                                   Index Cond: (ancestor_2 = 1)
                                   Buffers: shared hit=29
   ->  Merge Right Join  (cost=169565733.26..97549168801.28 rows=6491839610305 width=0) (actual time=1933.113..2294.894 rows=6005 loops=1)
         Merge Cond: (nodes_fts.node_id = t.node_id)
         Buffers: shared hit=3540868
         ->  Index Only Scan using nodes_fts_idx on nodes_fts  (cost=0.43..97055.96 rows=1701569 width=4) (actual time=0.077..271.313 rows=1598712 loops=1)
               Heap Fetches: 1598712
               Buffers: shared hit=16805
         ->  Materialize  (cost=169565732.84..173380940.81 rows=763041594 width=4) (actual time=1933.030..1933.903 rows=6005 loops=1)
               Buffers: shared hit=3524063
               ->  Sort  (cost=169565732.84..171473336.82 rows=763041594 width=4) (actual time=1933.026..1933.375 rows=6005 loops=1)
                     Sort Key: t.node_id
                     Sort Method: quicksort  Memory: 474kB
                     Buffers: shared hit=3524063
                     ->  CTE Scan on t  (cost=0.00..15260831.88 rows=763041594 width=4) (actual time=15.336..1932.145 rows=6005 loops=1)
                           Buffers: shared hit=3524063
 Planning time: 1.154 ms
 Execution time: 2295.801 ms
(40 rows)

1 个答案:

答案 0 :(得分:1)

估计的行数与实际数相差一个数量级:

  

在t上进行CTE扫描(费用= 0.00..15260831.88行= 763041594 宽度= 4)   (实际时间= 15.336..1932.145行= 6005 循环= 1)

当Postgres无法准确估计执行查询的特定方式与另一查询方式相比有多少工作时,它会生成效率低下的查询计划,因此即使所有数据都在RAM中,同一查询也会变慢

备份表时,转储不包含优化程序使用的统计信息,因此从转储中还原后,您需要等待autovacuum守护程序或手动运行'ANALYZE'。