我正在尝试调试生产速度慢的查询,但在我的开发机器上运行速度很快。我的开发框有一个prod数据库的快照,只有几天的时间,因此两个数据库的内容大致相同。
查询是:
select count(*) from big_table where search_column in ('something')
注意:
big_table
是snapshot materialized view,行数约为35M,每天刷新search_column
有一个b树索引。 explain analyze
的结果:
刺:
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=1119843.20..1119843.21 rows=1 width=0) (actual time=467388.276..467388.278 rows=1 loops=1)
-> Bitmap Heap Scan on big_table (cost=10432.55..1118804.45 rows=415497 width=0) (actual time=116891.126..466949.331 rows=210053 loops=1)
Recheck Cond: ((search_column)::text = 'something'::text)
-> Bitmap Index Scan on big_table_search_column_index (cost=0.00..10328.68 rows=415497 width=0) (actual time=8467.901..8467.901 rows=337164 loops=1)
Index Cond: ((search_column)::text = 'something'::text)
Total runtime: 467389.534 ms
(6 rows)
dev的:
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=524011.38..524011.39 rows=1 width=0) (actual time=209.852..209.852 rows=1 loops=1)
-> Bitmap Heap Scan on big_table (cost=5131.43..523531.22 rows=192064 width=0) (actual time=33.792..194.730 rows=209551 loops=1)
Recheck Cond: ((search_column)::text = 'something'::text)
-> Bitmap Index Scan on big_table_search_column_index (cost=0.00..5083.42 rows=192064 width=0) (actual time=27.568..27.568 rows=209551 loops=1)
Index Cond: ((search_column)::text = 'something'::text)
Total runtime: 209.938 ms
(6 rows)
和prod和dev的两个查询的实际结果分别是210053和209551行。
虽然这两个计划的结构是相同的,但是可以解释上述成本的差异,因为每个数据库中该表的行数大致相同?
关于@ bma的建议,这里是prod和dev的“膨胀”查询的结果以及相关的表/索引:
prod :
current_database | schemaname | tablename | tbloat | wastedbytes | iname | ibloat | wastedibytes
------------------+------------+---------------------------------+--------+-------------+---------------------------------------------------------------+--------+--------------
my_db | public | big_table | 1.6 | 7965433856 | big_table_search_column_index | 0.1 | 0
dev的:
current_database | schemaname | tablename | tbloat | wastedbytes | iname | ibloat | wastedibytes
------------------+------------+---------------------------------+--------+-------------+---------------------------------------------------------------+--------+--------------
my_db | public | big_table | 0.8 | 0 | big_table_search_column_index | 0.1 | 0
Voila,这里有所不同。
我已经运行vacuum analyze big_table;
,但这似乎与计数查询的运行时间没有任何显着差异。
bma建议SELECT name, current_setting(name), source FROM pg_settings WHERE source NOT IN ('default', 'override');
的结果:
刺:
name | current_setting | source
----------------------------+----------------------------------+----------------------
application_name | psql | client
DateStyle | ISO, MDY | configuration file
default_text_search_config | pg_catalog.english | configuration file
effective_cache_size | 6GB | configuration file
external_pid_file | /var/run/postgresql/9.1-main.pid | configuration file
listen_addresses | * | configuration file
log_line_prefix | %t | configuration file
log_timezone | localtime | environment variable
max_connections | 100 | configuration file
max_stack_depth | 2MB | environment variable
port | 5432 | configuration file
shared_buffers | 2GB | configuration file
ssl | on | configuration file
TimeZone | localtime | environment variable
unix_socket_directory | /var/run/postgresql | configuration file
(15 rows)
dev的:
name | current_setting | source
----------------------------+-------------------------+----------------------
application_name | psql | client
DateStyle | ISO, MDY | configuration file
default_text_search_config | pg_catalog.english | configuration file
effective_cache_size | 4GB | configuration file
lc_messages | en_US | configuration file
lc_monetary | en_US | configuration file
lc_numeric | en_US | configuration file
lc_time | en_US | configuration file
listen_addresses | * | configuration file
log_destination | syslog | configuration file
log_directory | ../var | configuration file
log_filename | postgresql-%Y-%m-%d.log | configuration file
log_line_prefix | %t | configuration file
log_statement | all | configuration file
log_timezone | Australia/Hobart | command line
logging_collector | on | configuration file
maintenance_work_mem | 512MB | configuration file
max_connections | 50 | configuration file
max_stack_depth | 2MB | environment variable
shared_buffers | 2GB | configuration file
ssl | off | configuration file
synchronous_commit | off | configuration file
TimeZone | Australia/Hobart | command line
timezone_abbreviations | Default | command line
work_mem | 100MB | configuration file
(25 rows)
答案 0 :(得分:1)
狂野猜测(对评论来说太长了......):由于数据分布,用于刷新mat视图的查询计划可能非常不同,导致mat视图以完全不同的方式填充
这最终可能会产生类似的位图索引扫描计划,但后者可以方便地访问您开发中的精选几个磁盘页面,而不是生产中的大量磁盘页面。
如果此引导对您有意义,您是否还可以发布用于实际创建/刷新mat视图的查询计划?如果它们差异很大(成本估算,计划等),请尝试在mat视图上创建聚簇索引(可能在search_column本身上),以查看它是否有任何重大差异。 (这样做之后别忘了分析。)
答案 1 :(得分:1)
如果您对此实体化视图表的大多数查询都受search_column
限制,我建议您在刷新后运行cluster big_table using big_table_search_column_index; reindex big_table;
。
这需要一些时间,并且会在运行时阻止此表,这会使磁盘上的表数据按此search_column排序。因此,所有被search_column
值限制的查询只需要检索有限数量的磁盘块,甚至可能需要从磁盘盘片上的有限位置检索。在cluster
之后,reindex会处理可能的索引膨胀。
我认为您的开发计算机有一个SSD驱动器,非常擅长从分散的位置检索数据。在生产中你可能有经典的旋转磁盘或磁盘,它们很糟糕。它也是最近创建的,因此没有膨胀(删除数据后留下的漏洞)。我认为这会导致几个数量级的减速。