为什么我在这个PostgreSQL查询中得到'Hash Join'和FTS?

时间:2013-01-18 17:20:26

标签: performance postgresql join hash indexing

我正在尝试优化以下方案:

措辞格式:我有2个表,alertsuser_devices;在user_devices中,我们会跟踪耦合到user_id的设备是否想要收到通知,并在alerts表中跟踪用户与通知程序的关系。基本上,任务是选择任何有警报的user_id,并允许向注册到它的任何设备发送通知。

表'警报',大约900k记录:

               Table "public.alerts"
   Column    |           Type           | Modifiers 
-------------+--------------------------+-----------
 id          | uuid                     | not null
 user_id     | uuid                     | 
 target_id   | uuid                     | 
 target_type | text                     | 
 added_on    | timestamp with time zone | 
 old_id      | text                     | 
Indexes:
    "alerts_pkey" PRIMARY KEY, btree (id)
    "one_alert_per_business_per_user" UNIQUE CONSTRAINT, btree (user_id, target_id)
    "addedon" btree (added_on)
    "targetid" btree (target_id)
    "userid" btree (user_id)
    "userid_targetid" btree (user_id, target_id)
Foreign-key constraints:
    "alerts_user_id_fkey" FOREIGN KEY (user_id) REFERENCES users(id)

表'user_devices',大约12k记录:

                Table "public.user_devices"
       Column        |           Type           | Modifiers 
---------------------+--------------------------+-----------
 id                  | uuid                     | not null
 user_id             | uuid                     | 
 device_id           | text                     | 
 device_token        | text                     | 
 push_notify_enabled | boolean                  | 
 device_type         | integer                  | 
 device_name         | text                     | 
 badge_count         | integer                  | 
 added_on            | timestamp with time zone | 
Indexes:
    "user_devices_pkey" PRIMARY KEY, btree (id)
    "push_notification" btree (push_notify_enabled)
    "user_id" btree (user_id)
    "user_id_push_notification" btree (user_id, push_notify_enabled)
Foreign-key constraints:
    "user_devices_user_id_fkey" FOREIGN KEY (user_id) REFERENCES users(id)

以下查询:

select COUNT(DISTINCT a.user_id) 
from alerts a 
  inner join user_devices ud on a.user_id = ud.user_id 
WHERE ud.push_notify_enabled = true;

大约需要3秒钟并制定以下计划:

explain select COUNT(DISTINCT a.user_id) from alerts a inner join user_devices ud on a.user_id = ud.user_id WHERE ud.push_notify_enabled = true;
                                     QUERY PLAN                                     
------------------------------------------------------------------------------------
 Aggregate  (cost=49777.32..49777.33 rows=1 width=16)
   ->  Hash Join  (cost=34508.97..48239.63 rows=615074 width=16)
         Hash Cond: (ud.user_id = a.user_id)
         ->  Seq Scan on user_devices ud  (cost=0.00..480.75 rows=9202 width=16)
               Filter: push_notify_enabled
         ->  Hash  (cost=20572.32..20572.32 rows=801732 width=16)
               ->  Seq Scan on alerts a  (cost=0.00..20572.32 rows=801732 width=16)

我缺少什么,有没有办法加快速度?

谢谢。

== edit ==

根据建议,尝试在连接中移动条件,没有区别:

=> explain select COUNT(DISTINCT a.user_id) from alerts a inner join user_devices ud on a.user_id = ud.user_id and ud.push_notify_enabled;
                                     QUERY PLAN                                     
------------------------------------------------------------------------------------
 Aggregate  (cost=49777.32..49777.33 rows=1 width=16)
   ->  Hash Join  (cost=34508.97..48239.63 rows=615074 width=16)
         Hash Cond: (ud.user_id = a.user_id)
         ->  Seq Scan on user_devices ud  (cost=0.00..480.75 rows=9202 width=16)
               Filter: push_notify_enabled
         ->  Hash  (cost=20572.32..20572.32 rows=801732 width=16)
               ->  Seq Scan on alerts a  (cost=0.00..20572.32 rows=801732 width=16)

那么,没办法摆脱2个FTS?如果我至少能够以某种方式使用“警报”表上的索引,那就太好了..

== edit ==

添加`EXPLAIN ANALYZE。

=> explain ANALYZE select COUNT(DISTINCT a.user_id) from alerts a inner join user_devices ud on a.user_id = ud.user_id and ud.push_notify_enabled;
                                                             QUERY PLAN                                                              
-------------------------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=49777.32..49777.33 rows=1 width=16) (actual time=5254.355..5254.356 rows=1 loops=1)
   ->  Hash Join  (cost=34508.97..48239.63 rows=615074 width=16) (actual time=1824.607..2863.635 rows=614768 loops=1)
         Hash Cond: (ud.user_id = a.user_id)
         ->  Seq Scan on user_devices ud  (cost=0.00..480.75 rows=9202 width=16) (actual time=0.048..16.784 rows=9186 loops=1)
               Filter: push_notify_enabled
         ->  Hash  (cost=20572.32..20572.32 rows=801732 width=16) (actual time=1824.229..1824.229 rows=801765 loops=1)
               Buckets: 4096  Batches: 32  Memory Usage: 990kB
               ->  Seq Scan on alerts a  (cost=0.00..20572.32 rows=801732 width=16) (actual time=0.047..878.429 rows=801765 loops=1)
 Total runtime: 5255.427 ms
(9 rows)

===编辑===

添加请求的配置。其中大部分是Ubuntu PG9.1默认值:

/etc/postgresql/9.1/main# cat postgresql.conf | grep -e "work_mem" -e "effective_cache" -e "shared_buff" -e "random_page_c"
shared_buffers = 24MB           # min 128kB
#work_mem = 1MB             # min 64kB
#maintenance_work_mem = 16MB        # min 1MB
#wal_buffers = -1           # min 32kB, -1 sets based on shared_buffers
#random_page_cost = 4.0         # same scale as above
#effective_cache_size = 128MB

2 个答案:

答案 0 :(得分:1)

如评论中所述,真正的生猪是alerts表的完整扫描。从逻辑上讲,对于给定的用户ID,alerts中的任何和所有记录都可能与该用户ID匹配。

您有一个可能限制扫描的条件:push_notify_enabled;你不需要行false的行。但是您在此列上缺少索引,因此alerts上的完整扫描仍然是加入这两个表的最快方式。

如果您的Postgres版本支持,请尝试在push_notify_enabled上使用位图索引。 (显然,2值列上的btree索引并不好。)

要加快查询速度,您必须在alerts中限制要扫描的行数,即在alerts的某个索引列上添加条件。如果索引具有足够的选择性,则可以进行索引扫描而不是完全扫描。

例如,如果有意义,您可以按目标ID或某个与日期相关的列进行过滤。

如果您有900k警报,这些警报都是活动的,并且可以在用户之间任意共享,您别无选择;可能添加RAM以保持alerts表始终缓存可能会有所帮助。 (添加硬件通常是最简单且最具成本效益的解决方案。)

AFAICT您只对与推送通知相关联的提醒感兴趣。如果具有推送通知的用户从不与没有推送通知的用户共享提醒,则可以通过此条件有效地拆分alerts

如果您有位图索引,可以将push_notify_enabled列移至alerts。否则,您可以尝试使用partitioning在该列上进行物理拆分。如果推送通知的警报数量明显低于总警报数量,则会扫描alerts的一小部分进行联接。

答案 1 :(得分:1)

用部分索引替换索引:

DROP INDEX    user_id_push_notification ;
CREATE INDEX    user_id_push_notification ON user_devices (user_id)
 WHERE push_notify_enabled =True
 ;

,并将random_page_cost设置为较低的值:

SET random_page_cost = 1.1;

为我造成Index Scan using push_notification on user_devices ud(<300毫秒)。因人而异。

警报上的seqscan似乎或多或少是不可避免的,因为你期望800K / 900K:= 88%行。只有当行大小非常​​大时,索引扫描才会有效,恕我直言。

更新:将users表添加到查询似乎强制进行三重索引扫描。 (但大约在同一时间)

explain  ANALYZE
select COUNT(DISTINCT a.user_id)
from alerts a
join user_devices ud on a.user_id = ud.user_id
join users us on a.user_id = us.id
WHERE ud.push_notify_enabled = true;