Question

我有一张名为“meta”的表，有220万条记录。该表包含3列：productId，key和value。键和值是包含产品元描述的两列。

以下查询需要2.6秒并返回676个结果（PostgreSQL 8.4.13，CentOS 6.4 64位）。此查询用于从特定过滤器（大小）中检索所有可能的元描述，其中用户已在其他两个过滤器（年份和来源）上过滤。

我尝试了这个主题的数组解决方案，但它只会让情况变得更糟：PostgreSQL IN operator with subquery poor performance

这两个子查询非常快（75毫秒和178毫秒），但组合它们会导致性能问题。有没有办法重写查询？

这是当前的查询：

SELECT DISTINCT ON(value) key, value 
FROM   "meta" 
WHERE  key = 'size'
    AND "productId" IN (SELECT "productId" 
        FROM   "meta" 
        WHERE  "value" = 'ibm'
            AND "key" = 'source' )
    AND "productId" IN (SELECT "productId" 
        FROM  "meta"
        WHERE "value" >= '1920'
            AND "value" <= '2010' 
            AND "key" = 'year' ) 
ORDER  BY value

使用以下EXPLAIN ANALYZE：

Unique  (cost=38829.46..38843.19 rows=564 width=15) (actual time=2674.474..2690.856 rows=676 loops=1)
  ->  Sort  (cost=38829.46..38836.32 rows=2745 width=15) (actual time=2674.471..2681.333 rows=66939 loops=1)
        Sort Key: public."meta".value
        Sort Method:  quicksort  Memory: 8302kB
        ->  Hash Join  (cost=32075.86..38672.69 rows=2745 width=15) (actual time=472.158..2472.002 rows=66939 loops=1)
              Hash Cond: (public."meta"."originalId" = public."meta"."productId")
              ->  Nested Loop  (cost=15079.41..21563.33 rows=13109 width=23) (actual time=113.873..1013.113 rows=104307 loops=1)
                    ->  HashAggregate  (cost=15079.41..15089.21 rows=980 width=4) (actual time=113.802..163.805 rows=105204 loops=1)
                          ->  Bitmap Heap Scan on "meta"  (cost=315.39..15051.42 rows=11196 width=4) (actual time=24.540..68.237 rows=105204 loops=1)
                                Recheck Cond: (((key)::text = 'source'::text) AND ((value)::text = 'KADASTER_WOII_RAF_USAAF'::text))
                                ->  Bitmap Index Scan on "productMetadataKeyValueIndex"  (cost=0.00..312.60 rows=11196 width=0) (actual time=23.506..23.506 rows=105204 loops=1)
                                      Index Cond: (((key)::text = 'source'::text) AND ((value)::text = 'ibm'::text))
                    ->  Index Scan using "idx_productId" on "meta"  (cost=0.00..6.59 rows=1 width=19) (actual time=0.006..0.008 rows=1 loops=105204)
                          Index Cond: (public."meta"."productId" = public."meta"."productId")
                          Filter: ((public."meta".key)::text = 'size'::text)
              ->  Hash  (cost=16954.58..16954.58 rows=3350 width=4) (actual time=358.214..358.214 rows=184571 loops=1)
                    ->  HashAggregate  (cost=16921.08..16954.58 rows=3350 width=4) (actual time=258.149..319.154 rows=184571 loops=1)
                          ->  Bitmap Heap Scan on "meta"  (cost=1172.62..16825.39 rows=38273 width=4) (actual time=86.725..167.110 rows=184571 loops=1)
                                Recheck Cond: (((key)::text = 'year'::text) AND ((value)::text >= '1920'::text) AND ((value)::text <= '2010'::text))
                                ->  Bitmap Index Scan on "productMetadataKeyIndex"  (cost=0.00..1163.05 rows=38273 width=0) (actual time=83.992..83.992 rows=184571 loops=1)
                                      Index Cond: (((key)::text = 'year'::text) AND ((value)::text >= '1920'::text) AND ((value)::text <= '2010'::text))
Total runtime: 2696.276 ms

定义索引：

idx_productId   CREATE INDEX "idx_productId" ON "meta" USING btree ("productId")    
productMetaUnique_id    CREATE UNIQUE INDEX "productMetaUnique_id" ON "meta" USING btree ("productId", key)     
productMetadataKeyIndex CREATE INDEX "productMetadataKeyIndex" ON "meta" USING btree (key)  
productMetadataKeyValueIndex    CREATE INDEX "productMetadataKeyValueIndex" ON "meta" USING btree (key, value)

Answer 1

首先，PostgreSQL 8.1.4是古董。升级，因为自8.2（8.3,8.4,9.0,9.1和9.2）以来的每个版本都对查询规划器进行了改进。

接下来，您可能会重写您的查询以使用联接和分组，并可能获得更好的计划。

select m1."value"
from meta as m1
join meta as m2 on m2."productId" = m1."productId"
               and m2."key" = 'source'
               and m2."value" = 'ibm'
join meta as m3 on m3."productId" = m1."productId"
               and m3."key" = 'year'
               and m3."value" between 1920 and 2010
where m1."key" = 'size'
group by m1."value"

后者可能会使用(key, product_id)和(product_id, key, value)上的索引进行PG 9.2中的仅索引扫描，从而完全避免表查找。

接下来，您应该直接将数据放入product表中。如果您正在查询它，它可能首先不属于元。

最后，如果确实希望将这些内容保存在元数据中，那么可能会出现这样的情况，即使用现有语句进行操作是值得的：

select val
from unnest(array['Known', 'Sizes', 'Go', 'Here']::text[]) as val
where exists (
    select 1
    from meta as m1
    join meta as m2 on m2."productId" = m1."productId"
                   and m2."key" = 'source'
                   and m2."value" = 'ibm'
    join meta as m3 on m3."productId" = m1."productId"
                   and m3."key" = 'year'
                   and m3."value" between 1920 and 2010
    where m1."key" = 'size'
      and m1."value" = val
  );

这样做可以免除昂贵的group by，sort和unique操作。

Postgres两个子查询性能不佳[EAV模型]

1 个答案: