我正在尝试优化以下方案:
措辞格式:我有2个表,alerts
和user_devices
;在user_devices
中,我们会跟踪耦合到user_id
的设备是否想要收到通知,并在alerts
表中跟踪用户与通知程序的关系。基本上,任务是选择任何有警报的user_id
,并允许向注册到它的任何设备发送通知。
表'警报',大约900k记录:
Table "public.alerts"
Column | Type | Modifiers
-------------+--------------------------+-----------
id | uuid | not null
user_id | uuid |
target_id | uuid |
target_type | text |
added_on | timestamp with time zone |
old_id | text |
Indexes:
"alerts_pkey" PRIMARY KEY, btree (id)
"one_alert_per_business_per_user" UNIQUE CONSTRAINT, btree (user_id, target_id)
"addedon" btree (added_on)
"targetid" btree (target_id)
"userid" btree (user_id)
"userid_targetid" btree (user_id, target_id)
Foreign-key constraints:
"alerts_user_id_fkey" FOREIGN KEY (user_id) REFERENCES users(id)
表'user_devices',大约12k记录:
Table "public.user_devices"
Column | Type | Modifiers
---------------------+--------------------------+-----------
id | uuid | not null
user_id | uuid |
device_id | text |
device_token | text |
push_notify_enabled | boolean |
device_type | integer |
device_name | text |
badge_count | integer |
added_on | timestamp with time zone |
Indexes:
"user_devices_pkey" PRIMARY KEY, btree (id)
"push_notification" btree (push_notify_enabled)
"user_id" btree (user_id)
"user_id_push_notification" btree (user_id, push_notify_enabled)
Foreign-key constraints:
"user_devices_user_id_fkey" FOREIGN KEY (user_id) REFERENCES users(id)
以下查询:
select COUNT(DISTINCT a.user_id)
from alerts a
inner join user_devices ud on a.user_id = ud.user_id
WHERE ud.push_notify_enabled = true;
大约需要3秒钟并制定以下计划:
explain select COUNT(DISTINCT a.user_id) from alerts a inner join user_devices ud on a.user_id = ud.user_id WHERE ud.push_notify_enabled = true;
QUERY PLAN
------------------------------------------------------------------------------------
Aggregate (cost=49777.32..49777.33 rows=1 width=16)
-> Hash Join (cost=34508.97..48239.63 rows=615074 width=16)
Hash Cond: (ud.user_id = a.user_id)
-> Seq Scan on user_devices ud (cost=0.00..480.75 rows=9202 width=16)
Filter: push_notify_enabled
-> Hash (cost=20572.32..20572.32 rows=801732 width=16)
-> Seq Scan on alerts a (cost=0.00..20572.32 rows=801732 width=16)
我缺少什么,有没有办法加快速度?
谢谢。
== edit ==
根据建议,尝试在连接中移动条件,没有区别:
=> explain select COUNT(DISTINCT a.user_id) from alerts a inner join user_devices ud on a.user_id = ud.user_id and ud.push_notify_enabled;
QUERY PLAN
------------------------------------------------------------------------------------
Aggregate (cost=49777.32..49777.33 rows=1 width=16)
-> Hash Join (cost=34508.97..48239.63 rows=615074 width=16)
Hash Cond: (ud.user_id = a.user_id)
-> Seq Scan on user_devices ud (cost=0.00..480.75 rows=9202 width=16)
Filter: push_notify_enabled
-> Hash (cost=20572.32..20572.32 rows=801732 width=16)
-> Seq Scan on alerts a (cost=0.00..20572.32 rows=801732 width=16)
那么,没办法摆脱2个FTS?如果我至少能够以某种方式使用“警报”表上的索引,那就太好了..
== edit ==
添加`EXPLAIN ANALYZE。
=> explain ANALYZE select COUNT(DISTINCT a.user_id) from alerts a inner join user_devices ud on a.user_id = ud.user_id and ud.push_notify_enabled;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=49777.32..49777.33 rows=1 width=16) (actual time=5254.355..5254.356 rows=1 loops=1)
-> Hash Join (cost=34508.97..48239.63 rows=615074 width=16) (actual time=1824.607..2863.635 rows=614768 loops=1)
Hash Cond: (ud.user_id = a.user_id)
-> Seq Scan on user_devices ud (cost=0.00..480.75 rows=9202 width=16) (actual time=0.048..16.784 rows=9186 loops=1)
Filter: push_notify_enabled
-> Hash (cost=20572.32..20572.32 rows=801732 width=16) (actual time=1824.229..1824.229 rows=801765 loops=1)
Buckets: 4096 Batches: 32 Memory Usage: 990kB
-> Seq Scan on alerts a (cost=0.00..20572.32 rows=801732 width=16) (actual time=0.047..878.429 rows=801765 loops=1)
Total runtime: 5255.427 ms
(9 rows)
===编辑===
添加请求的配置。其中大部分是Ubuntu PG9.1默认值:
/etc/postgresql/9.1/main# cat postgresql.conf | grep -e "work_mem" -e "effective_cache" -e "shared_buff" -e "random_page_c"
shared_buffers = 24MB # min 128kB
#work_mem = 1MB # min 64kB
#maintenance_work_mem = 16MB # min 1MB
#wal_buffers = -1 # min 32kB, -1 sets based on shared_buffers
#random_page_cost = 4.0 # same scale as above
#effective_cache_size = 128MB
答案 0 :(得分:1)
如评论中所述,真正的生猪是alerts
表的完整扫描。从逻辑上讲,对于给定的用户ID,alerts
中的任何和所有记录都可能与该用户ID匹配。
您有一个可能限制扫描的条件:push_notify_enabled
;你不需要行false
的行。但是您在此列上缺少索引,因此alerts
上的完整扫描仍然是加入这两个表的最快方式。
如果您的Postgres版本支持,请尝试在 push_notify_enabled
上使用位图索引。 (显然,2值列上的btree索引并不好。)
要加快查询速度,您必须在alerts
中限制要扫描的行数,即在alerts
的某个索引列上添加条件。如果索引具有足够的选择性,则可以进行索引扫描而不是完全扫描。
例如,如果有意义,您可以按目标ID或某个与日期相关的列进行过滤。
如果您有900k警报,这些警报都是活动的,并且可以在用户之间任意共享,您别无选择;可能添加RAM以保持alerts
表始终缓存可能会有所帮助。 (添加硬件通常是最简单且最具成本效益的解决方案。)
AFAICT您只对与推送通知相关联的提醒感兴趣。如果具有推送通知的用户从不与没有推送通知的用户共享提醒,则可以通过此条件有效地拆分alerts
。
如果您有位图索引,可以将push_notify_enabled
列移至alerts
。否则,您可以尝试使用partitioning在该列上进行物理拆分。如果推送通知的警报数量明显低于总警报数量,则会扫描alerts
的一小部分进行联接。
答案 1 :(得分:1)
用部分索引替换索引:
DROP INDEX user_id_push_notification ;
CREATE INDEX user_id_push_notification ON user_devices (user_id)
WHERE push_notify_enabled =True
;
,并将random_page_cost设置为较低的值:
SET random_page_cost = 1.1;
为我造成Index Scan using push_notification on user_devices ud
(<300毫秒)。因人而异。
警报上的seqscan似乎或多或少是不可避免的,因为你期望800K / 900K:= 88%行。只有当行大小非常大时,索引扫描才会有效,恕我直言。
更新:将users表添加到查询似乎强制进行三重索引扫描。 (但大约在同一时间)
explain ANALYZE
select COUNT(DISTINCT a.user_id)
from alerts a
join user_devices ud on a.user_id = ud.user_id
join users us on a.user_id = us.id
WHERE ud.push_notify_enabled = true;