我想加快以下SQL的速度（费用为19685.75）。我可以在WHERE语句中索引具有多个复杂嵌套AND条件并与OR结合的该sql吗？
SELECT DISTINCT
ON ("crawler_url"."url") U0."id"
FROM "characteristics_text" U0 LEFT OUTER
JOIN "characteristics_text_urls"
ON (U0."id" = "characteristics_text_urls"."text_id") LEFT OUTER
JOIN "crawler_url"
ON ("characteristics_text_urls"."url_id" = "crawler_url"."id")
WHERE (
(
U0."publication_date" BETWEEN '2018-01-01' AND '2018-12-31'
AND EXTRACT('month' FROM U0."publication_date") = 10
)
OR
(
U0."publication_date" IS NULL
AND U0."lastmod" BETWEEN '2018-01-01' AND '2018-12-31'
AND EXTRACT('month' FROM U0."lastmod") = 10
)
OR
(
U0."publication_date" IS NULL
AND U0."lastmod" IS NULL
AND U0."created_at" BETWEEN '2018-01-01 00:00:00+08:00' AND '2018-12-31 23:59:59.999999+08:00'
AND EXTRACT('month' FROM U0."created_at" AT TIME ZONE 'Asia/Hong_Kong') = 10
)
OR
(
U0."publication_date" >= '2018-08-01'
AND U0."publication_date" < '2018-10-31'
)
OR
(
U0."publication_date" IS NULL
AND U0."lastmod" >='2018-08-01'
AND U0."lastmod" < '2018-10-31'
)
OR
(
U0."publication_date" IS NULL
AND U0."lastmod" IS NULL
AND U0."created_at" >= '2018-07-31 16:00:00+00:00'
AND U0."created_at" < '2018-10-30 16:00:00+00:00'
)
)
ORDER BY "crawler_url"."url" ASC, U0."created_at" DESC
表文本包含以下字段和索引（其他一些字段未显示）
Table "public.characteristics_text"
Column | Type | Modifiers
------------------------+--------------------------+-------------------------------------------------------------------
id | integer | not null default nextval('characteristics_text_id_seq'::regclass)
text | text |
created_at | timestamp with time zone | not null
lastmod | date |
publication_date | date |
Indexes:
"characteristics_text_pkey" PRIMARY KEY, btree (id)
"characteristics_text_fde81f11" btree (created_at)
"characteristics_text_lastmod_3bff34c2_uniq" btree (lastmod)
"characteristics_text_publication_date_772c1bda_uniq" btree (publication_date)
"characteristics_text_publication_date_c6311385_idx" btree (publication_date, lastmod, created_at)
我为created_at，lastmod和publication_date添加了三个单一索引；以及这些字段的一个多列索引。
但是在postgres EXPAIN查询中，此where子句仍在使用 Seq扫描，而不是索引扫描。
-> Seq Scan on characteristics_text u0 (cost=0.00..19685.75 rows=14535 width=12)
Filter: (
(
(publication_date >= '2018-01-01'::date) AND
(publication_date <= '2018-12-31'::date) AND
(
date_part(
'month'::text, (publication_date)::timestamp without time zone
) = 10::double precision)
) OR
((publication_date IS NULL) AND (lastmod >= '2018-01-01'::date) AND (lastmod <= '2018-12-31'::date) AND (date_part('month'::text, (lastmod)::timestamp without time zone) = 10::double precision)) OR ((publication_date IS NULL) AND (lastmod IS NULL) AND (created_at >= '2017-12-31 16:00:00+00'::timestamp with time zone) AND (created_at <= '2018-12-31 15:59:59.999999+00'::timestamp with time zone) AND (date_part('month'::text, timezone('Asia/Hong_Kong'::text, created_at)) = 10::double precision)) OR ((publication_date >= '2018-08-01'::date) AND (publication_date < '2018-10-31'::date)) OR ((publication_date IS NULL) AND (lastmod >= '2018-08-01'::date) AND (lastmod < '2018-10-31'::date)) OR ((publication_date IS NULL) AND (lastmod IS NULL) AND (created_at >= '2018-07-31 16:00:00+00'::timestamp with time zone) AND (created_at < '2018-10-30 16:00:00+00'::timestamp with time zone))
)
我的问题是：
1.是否可以使postgres对此复杂的SELECT子句使用索引扫描？
2.是否需要为每个AND子句创建一个多列索引？例如，创建索引(publication_date, lastmod)的原因是什么？
(
U0."publication_date" IS NULL
AND U0."lastmod" BETWEEN '2018-01-01' AND '2018-12-31'
AND EXTRACT('month' FROM U0."lastmod") = 10
)
已更新4nov2018
当我尝试通过逐个测试字段来最小化查询时，字段publication_date和last_mod分别触发索引扫描，而created_at不能：
是因为created_at是时间戳吗？但是索引为什么不能用于时间戳？
explain SELECT DISTINCT
ON ("crawler_url"."url") U0."id"
FROM "characteristics_text" U0 LEFT OUTER
JOIN "characteristics_text_urls"
ON (U0."id" = "characteristics_text_urls"."text_id") LEFT OUTER
JOIN "crawler_url"
ON ("characteristics_text_urls"."url_id" = "crawler_url"."id")
WHERE (
(
U0."created_at" BETWEEN '2018-01-01 00:00:00+08:00' AND '2018-12-31 23:59:59.999999+08:00'
AND EXTRACT('month' FROM U0."created_at" AT TIME ZONE 'Asia/Hong_Kong') = 10
)
)
ORDER BY "crawler_url"."url" ASC, U0."created_at" DESC
Unique (cost=18004.05..18006.01 rows=393 width=86)
-> Sort (cost=18004.05..18005.03 rows=393 width=86)
Sort Key: crawler_url.url, u0.created_at
-> Nested Loop Left Join (cost=0.71..17987.11 rows=393 width=86)
-> Nested Loop Left Join (cost=0.42..17842.25 rows=393 width=16)
-> Seq Scan on characteristics_text u0 (cost=0.00..15467.37 rows=393 width=12)
Filter: ((created_at >= '2017-12-31 16:00:00+00'::timestamp with time zone) AND (created_at <= '2018-12-31 15:59:59.999999+00'::timestamp with time zone) AND (date_part('month'::text, timezone('Asia/Hong_Kong'::text, created_at)) = 10::double precision))
-> Index Scan using characteristics_text_urls_65eb77fe on characteristics_text_urls (cost=0.42..6.03 rows=1 width=8)
Index Cond: (u0.id = text_id)
-> Index Scan using crawler_url_pkey on crawler_url (cost=0.29..0.36 rows=1 width=78)
Index Cond: (characteristics_text_urls.url_id = id)
publication_date似乎触发了索引扫描：
(
U0."publication_date" IS NULL
AND U0."lastmod" >='2018-08-01'
AND U0."lastmod" < '2018-10-31'
)
Unique (cost=17053.26..17085.63 rows=6473 width=86)
-> Sort (cost=17053.26..17069.44 rows=6473 width=86)
Sort Key: crawler_url.url, u0.created_at
-> Nested Loop Left Join (cost=11130.73..16643.51 rows=6473 width=86)
-> Hash Right Join (cost=11130.44..14257.63 rows=6473 width=16)
Hash Cond: (characteristics_text_urls.text_id = u0.id)
-> Seq Scan on characteristics_text_urls (cost=0.00..1858.01 rows=120601 width=8)
-> Hash (cost=11049.53..11049.53 rows=6473 width=12)
-> Bitmap Heap Scan on characteristics_text u0 (cost=186.95..11049.53 rows=6473 width=12)
Recheck Cond: ((publication_date IS NULL) AND (lastmod >= '2018-08-01'::date) AND (lastmod < '2018-10-31'::date))
-> Bitmap Index Scan on characteristics_text_publication_date_c6311385_idx (cost=0.00..185.33 rows=6473 width=0)
Index Cond: ((publication_date IS NULL) AND (lastmod >= '2018-08-01'::date) AND (lastmod < '2018-10-31'::date))
-> Index Scan using crawler_url_pkey on crawler_url (cost=0.29..0.36 rows=1 width=78)
Index Cond: (characteristics_text_urls.url_id = id)
答案 0 :(得分：2)
好的，全表扫描（seq_scan）实际上比多个索引扫描要快。这取决于您的过滤条件的特定“选择性”。
首先，您的WHERE子句具有六个OR设置的过滤条件。这意味着，如果要使用索引，PostgreSQL将需要使用6次，然后执行“索引OR”以合并结果。那可能并不便宜。
因此，首先，您需要知道6种过滤条件中的每一种的预期选择性。即，与表具有的总行数相比，选择了多少行。做吧一些简单的SQL查询将为您提供答案。在此处发布答案。
现在，如果所有六个选择性的总和超过5％，则全表扫描（您现在拥有的算法）会更快。不用担心索引。
否则，以下索引可以提供帮助：
create index ix1 on characteristics_text (
publication_date,
lastmod,
created_at,
1);
答案 1 :(得分：0)
我怀疑您将获得一个有用的索引。您可能考虑做的是将此查询分为4或5个部分，然后使用UNION将结果再次合并在一起。 （UNION将删除重复项，WHILE UNION ALL将返回所有行。）
UNION是一个相当昂贵的操作，因此需要考虑返回的行数。如果删除足够数量的行，使用索引可能会比UNION失去的效率更高。如果返回许多行，则您当前的表单将与将要获得的表单差不多。
答案 2 :(得分：0)
2018年全年有100,000条记录中的60％，这使得数据库使用了seq扫描。从整个年度变为只有一个月即可进行索引扫描。
AND U0."created_at" >= '2018-10-01 00:00:00+00:00'
AND U0."created_at" <= '2018-10-31 23:59:59.999999+00:00')
完整的SQL：
SELECT DISTINCT
ON ("crawler_url"."url") U0."id"
FROM "characteristics_text" U0 LEFT OUTER
JOIN "characteristics_text_urls"
ON (U0."id" = "characteristics_text_urls"."text_id") LEFT OUTER
JOIN "crawler_url"
ON ("characteristics_text_urls"."url_id" = "crawler_url"."id")
WHERE (
(U0."publication_date" >= '2018-10-01'
AND U0."publication_date" <= '2018-11-01')
OR (U0."publication_date" IS NULL
AND U0."lastmod" >= '2018-10-01'
AND U0."lastmod" <= '2018-11-01'
)
OR
(U0."publication_date" IS NULL
AND U0."lastmod" IS NULL
AND U0."created_at" >= '2018-10-01 00:00:00+00:00'
AND U0."created_at" <= '2018-10-31 23:59:59.999999+00:00')
OR
(U0."publication_date" >= '2018-08-01'
AND U0."publication_date" < '2018-10-31')
OR
(U0."publication_date" IS NULL
AND U0."lastmod" >= '2018-08-01'
AND U0."lastmod" < '2018-10-31')
OR
(U0."publication_date" IS NULL
AND U0."lastmod" IS NULL
AND U0."created_at" >= '2018-07-31 16:00:00+00:00'
AND U0."created_at" < '2018-10-30 16:00:00+00:00')
)
ORDER BY "crawler_url"."url" ASC
EXPLAIN语句显示每个AND条件的索引扫描，因此总计有6次索引扫描。
Unique (cost=22885.16..22962.39 rows=15446 width=88)
-> Sort (cost=22885.16..22923.77 rows=15446 width=88)
Sort Key: crawler_url.url
-> Hash Right Join (cost=18669.29..21068.51 rows=15446 width=88)
Hash Cond: (crawler_url.id = characteristics_text_urls.url_id)
-> Seq Scan on crawler_url (cost=0.00..1691.88 rows=55288 width=88)
-> Hash (cost=18476.21..18476.21 rows=15446 width=8)
-> Hash Right Join (cost=14982.09..18476.21 rows=15446 width=8)
Hash Cond: (characteristics_text_urls.text_id = u0.id)
-> Seq Scan on characteristics_text_urls (cost=0.00..1907.25 rows=115525 width=8)
-> Hash (cost=14789.01..14789.01 rows=15446 width=4)
-> Bitmap Heap Scan on characteristics_text u0 (cost=516.57..14789.01 rows=15446 width=4)
Recheck Cond: (((publication_date >= '2018-10-01'::date) AND (publication_date <= '2018-11-01'::date)) OR ((publication_date IS NULL) AND (lastmod >= '2018-10-01'::date) AND (lastmod <= '2018-11-01'::date)) OR ((publication_date IS NULL) AND (lastmod IS NULL) AND (created_at >= '2018-10-01 00:00:00+00'::timestamp with time zone) AND (created_at <= '2018-10-31 23:59:59.999999+00'::timestamp with time zone)) OR ((publication_date >= '2018-08-01'::date) AND (publication_date < '2018-10-31'::date)) OR ((publication_date IS NULL) AND (lastmod >= '2018-08-01'::date) AND (lastmod < '2018-10-31'::date)) OR ((publication_date IS NULL) AND (lastmod IS NULL) AND (created_at >= '2018-07-31 16:00:00+00'::timestamp with time zone) AND (created_at < '2018-10-30 16:00:00+00'::timestamp with time zone)))
-> BitmapOr (cost=516.57..516.57 rows=16081 width=0)
-> Bitmap Index Scan on characteristics_text_publication_date_772c1bda_uniq (cost=0.00..4.53 rows=11 width=0)
Index Cond: ((publication_date >= '2018-10-01'::date) AND (publication_date <= '2018-11-01'::date))
-> Bitmap Index Scan on characteristics_text_publication_date_c6311385_idx (cost=0.00..6.49 rows=166 width=0)
Index Cond: ((publication_date IS NULL) AND (lastmod >= '2018-10-01'::date) AND (lastmod <= '2018-11-01'::date))
-> Bitmap Index Scan on characteristics_text_publication_date_c6311385_idx (cost=0.00..14.61 rows=413 width=0)
Index Cond: ((publication_date IS NULL) AND (lastmod IS NULL) AND (created_at >= '2018-10-01 00:00:00+00'::timestamp with time zone) AND (created_at <= '2018-10-31 23:59:59.999999+00'::timestamp with time zone))
-> Bitmap Index Scan on characteristics_text_publication_date_772c1bda_uniq (cost=0.00..74.61 rows=3419 width=0)
Index Cond: ((publication_date >= '2018-08-01'::date) AND (publication_date < '2018-10-31'::date))
-> Bitmap Index Scan on characteristics_text_publication_date_c6311385_idx (cost=0.00..108.20 rows=3503 width=0)
Index Cond: ((publication_date IS NULL) AND (lastmod >= '2018-08-01'::date) AND (lastmod < '2018-10-31'::date))
-> Bitmap Index Scan on characteristics_text_publication_date_c6311385_idx (cost=0.00..284.95 rows=8569 width=0)
Index Cond: ((publication_date IS NULL) AND (lastmod IS NULL) AND (created_at >= '2018-07-31 16:00:00+00'::timestamp with time zone) AND (created_at < '2018-10-30 16:00:00+00'::timestamp with time zone))