Question

我有一个包含3个表的数据库，masterInfo，primDescT，secDescT。

CREATE TABLE masterInfo (id INTEGER PRIMARY KEY AUTOINCREMENT,
primDescId INTEGER,
secDescId INTEGER,
category INTEGER,
UNIQUE(primDescId, secDescId, category));

CREATE TABLE primDescT (id INTEGER PRIMARY KEY,
primDesc nvarchar(512));

CREATE TABLE secDescT (id INTEGER PRIMARY KEY,
secDesc nvarchar(512));

INSERT INTO primDescT VALUES(1,'XXXX');
INSERT INTO primDescT VALUES(2,'YYYY');
INSERT INTO primDescT VALUES(3,'ZZZZ');
INSERT INTO primDescT VALUES(4,'SSSS');

INSERT INTO secDescT VALUES(1,'AAA');
INSERT INTO secDescT VALUES(2,'BBB');
INSERT INTO secDescT VALUES(3,'CCC');

INSERT INTO masterInfo VALUES(1,1,1,1);
INSERT INTO masterInfo VALUES(2,2,2,2);
INSERT INTO masterInfo VALUES(3,3,1,1);
INSERT INTO masterInfo VALUES(4,4,3,2);

表，masterInfo有1765137行，primDescT中有312210行，secDescT中有105458行。

我使用以下查询来获取结果。

SELECT m.id AS pId, 
primDesc AS pDescr, secDesc AS sDescr, category   AS category 
FROM masterInfo m
INNER JOIN primDescT ON primDescT.id = m.primDescId
INNER JOIN secDescT ON secDescT.id = m.secDescId
WHERE m.category IN ('1','2') ORDER BY pDescr ASC  LIMIT 100 OFFSET 0

以上查询需要8秒才能响应。

但如果我将偏移设置为1756300，则需要53秒。

SELECT m.id AS pId, 
primDesc AS pDescr, secDesc AS sDescr, category   AS category 
FROM masterInfo m
INNER JOIN primDescT ON primDescT.id = m.primDescId
INNER JOIN secDescT ON secDescT.id = m.secDescId
WHERE m.category IN ('1','2') ORDER BY pDescr ASC  LIMIT 100 OFFSET 1756300

如何优化上述查询以在3秒内获取？

Answer 1

这些查询的问题是ORDER BY：必须先计算所有结果，然后数据库才能确定100或1756400最小的结果。 EXPLAIN QUERY PLAN输出：

0,0,0,SCAN TABLE masterInfo AS m
0,1,1,SEARCH TABLE primDescT USING INTEGER PRIMARY KEY (rowid=?)
0,2,2,SEARCH TABLE secDescT USING INTEGER PRIMARY KEY (rowid=?)
0,0,0,USE TEMP B-TREE FOR ORDER BY

要删除显式排序步骤，您必须索引该列：

CREATE INDEX pd ON primDescT(primDesc);

并且您必须强制数据库使用此索引（默认情况下，SQLite在估计查询成本时忽略LIMIT，如果您希望所有结果，而不是使用pd索引会更快）：

SELECT ...
FROM masterInfo m
INNER JOIN primDescT INDEXED BY pd ON primDescT.id = m.primDescId
--                   ^^^^^^^^^^^^^
INNER JOIN secDescT ON secDescT.id = m.secDescId
WHERE ...
ORDER BY pDescr ASC
LIMIT 100 OFFSET ...;

0,0,1,SCAN TABLE primDescT USING COVERING INDEX pd
0,1,0,SEARCH TABLE masterInfo AS m USING COVERING INDEX sqlite_autoindex_masterInfo_1 (primDescId=?)
0,2,2,SEARCH TABLE secDescT USING INTEGER PRIMARY KEY (rowid=?)

一个大的OFFSET值总是很慢;数据库必须计算并丢弃所有这些行。

如果您正在使用分页，则可以使用排序列上的查找替换OFFSET;这要求您保存上一页的最后一个值：

SELECT ...
FROM masterInfo m
INNER JOIN primDescT INDEXED BY pd ON primDescT.id = m.primDescId
INNER JOIN secDescT ON secDescT.id = m.secDescId
WHERE primDesc > :LastValue
--    ^^^^^^^^^^^^^^^^^^^^^
  AND ...
ORDER BY pDescr ASC
LIMIT 100 /* no offset */;

sqlite中内连接的查询优化

1 个答案: