Postgresql查询计划的差异

时间:2013-10-03 01:51:57

标签: postgresql database-performance sql-execution-plan

我正在尝试调试生产速度慢的查询,但在我的开发机器上运行速度很快。我的开发框有一个prod数据库的快照,只有几天的时间,因此两个数据库的内容大致相同。

查询是:

select count(*) from big_table where search_column in ('something')

注意:

  • big_tablesnapshot materialized view,行数约为35M,每天刷新
  • search_column有一个b树索引。
  • prod在ubuntu上是9.1
  • OS上的
  • dev是9.0

查询计划

explain analyze的结果:

QUERY PLAN                                                                                    
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=1119843.20..1119843.21 rows=1 width=0) (actual time=467388.276..467388.278 rows=1 loops=1)
   ->  Bitmap Heap Scan on big_table  (cost=10432.55..1118804.45 rows=415497 width=0) (actual time=116891.126..466949.331 rows=210053 loops=1)
         Recheck Cond: ((search_column)::text = 'something'::text)
         ->  Bitmap Index Scan on big_table_search_column_index  (cost=0.00..10328.68 rows=415497 width=0) (actual time=8467.901..8467.901 rows=337164 loops=1)
               Index Cond: ((search_column)::text = 'something'::text)
 Total runtime: 467389.534 ms
(6 rows)

dev的

QUERY PLAN                                                                                 
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=524011.38..524011.39 rows=1 width=0) (actual time=209.852..209.852 rows=1 loops=1)
   ->  Bitmap Heap Scan on big_table  (cost=5131.43..523531.22 rows=192064 width=0) (actual time=33.792..194.730 rows=209551 loops=1)
         Recheck Cond: ((search_column)::text = 'something'::text)
         ->  Bitmap Index Scan on big_table_search_column_index  (cost=0.00..5083.42 rows=192064 width=0) (actual time=27.568..27.568 rows=209551 loops=1)
               Index Cond: ((search_column)::text = 'something'::text)
 Total runtime: 209.938 ms
(6 rows)

和prod和dev的两个查询的实际结果分别是210053和209551行。

虽然这两个计划的结构是相同的,但是可以解释上述成本的差异,因为每个数据库中该表的行数大致相同?

鼓胀症

关于@ bma的建议,这里是prod和dev的“膨胀”查询的结果以及相关的表/索引:

prod

current_database | schemaname |            tablename            | tbloat | wastedbytes |                             iname                             | ibloat | wastedibytes 
------------------+------------+---------------------------------+--------+-------------+---------------------------------------------------------------+--------+--------------
my_db            | public     | big_table                       |    1.6 |  7965433856 | big_table_search_column_index                                 |    0.1 |            0

dev的

current_database | schemaname |            tablename            | tbloat | wastedbytes |                             iname                             | ibloat | wastedibytes 
------------------+------------+---------------------------------+--------+-------------+---------------------------------------------------------------+--------+--------------
my_db            | public     | big_table                       |    0.8 |           0 | big_table_search_column_index                                 |    0.1 |            0

Voila,这里有所不同。

我已经运行vacuum analyze big_table;,但这似乎与计数查询的运行时间没有任何显着差异。

配置

bma建议SELECT name, current_setting(name), source FROM pg_settings WHERE source NOT IN ('default', 'override');的结果:

            name            |         current_setting          |        source        
----------------------------+----------------------------------+----------------------
 application_name           | psql                             | client
 DateStyle                  | ISO, MDY                         | configuration file
 default_text_search_config | pg_catalog.english               | configuration file
 effective_cache_size       | 6GB                              | configuration file
 external_pid_file          | /var/run/postgresql/9.1-main.pid | configuration file
 listen_addresses           | *                                | configuration file
 log_line_prefix            | %t                               | configuration file
 log_timezone               | localtime                        | environment variable
 max_connections            | 100                              | configuration file
 max_stack_depth            | 2MB                              | environment variable
 port                       | 5432                             | configuration file
 shared_buffers             | 2GB                              | configuration file
 ssl                        | on                               | configuration file
 TimeZone                   | localtime                        | environment variable
 unix_socket_directory      | /var/run/postgresql              | configuration file
(15 rows)

dev的

            name            |     current_setting     |        source        
----------------------------+-------------------------+----------------------
 application_name           | psql                    | client
 DateStyle                  | ISO, MDY                | configuration file
 default_text_search_config | pg_catalog.english      | configuration file
 effective_cache_size       | 4GB                     | configuration file
 lc_messages                | en_US                   | configuration file
 lc_monetary                | en_US                   | configuration file
 lc_numeric                 | en_US                   | configuration file
 lc_time                    | en_US                   | configuration file
 listen_addresses           | *                       | configuration file
 log_destination            | syslog                  | configuration file
 log_directory              | ../var                  | configuration file
 log_filename               | postgresql-%Y-%m-%d.log | configuration file
 log_line_prefix            | %t                      | configuration file
 log_statement              | all                     | configuration file
 log_timezone               | Australia/Hobart        | command line
 logging_collector          | on                      | configuration file
 maintenance_work_mem       | 512MB                   | configuration file
 max_connections            | 50                      | configuration file
 max_stack_depth            | 2MB                     | environment variable
 shared_buffers             | 2GB                     | configuration file
 ssl                        | off                     | configuration file
 synchronous_commit         | off                     | configuration file
 TimeZone                   | Australia/Hobart        | command line
 timezone_abbreviations     | Default                 | command line
 work_mem                   | 100MB                   | configuration file
(25 rows)

2 个答案:

答案 0 :(得分:1)

狂野猜测(对评论来说太长了......):由于数据分布,用于刷新mat视图的查询计划可能非常不同,导致mat视图以完全不同的方式填充

这最终可能会产生类似的位图索引扫描计划,但后者可以方便地访问您开发中的精选几个磁盘页面,而不是生产中的大量磁盘页面。

如果此引导对您有意义,您是否还可以发布用于实际创建/刷新mat视图的查询计划?如果它们差异很大(成本估算,计划等),请尝试在mat视图上创建聚簇索引(可能在search_column本身上),以查看它是否有任何重大差异。 (这样做之后别忘了分析。)

答案 1 :(得分:1)

如果您对此实体化视图表的大多数查询都受search_column限制,我建议您在刷新后运行cluster big_table using big_table_search_column_index; reindex big_table;

这需要一些时间,并且会在运行时阻止此表,这会使磁盘上的表数据按此search_column排序。因此,所有被search_column值限制的查询只需要检索有限数量的磁盘块,甚至可能需要从磁盘盘片上的有限位置检索。在cluster之后,reindex会处理可能的索引膨胀。

我认为您的开发计算机有一个SSD驱动器,非常擅长从分散的位置检索数据。在生产中你可能有经典的旋转磁盘或磁盘,它们很糟糕。它也是最近创建的,因此没有膨胀(删除数据后留下的漏洞)。我认为这会导致几个数量级的减速。