通过计算列排序来优化SQLite查询?

时间:2015-09-26 01:48:33

标签: python sqlite

我正在使用sqlite查询来使用某个公式查找具有最佳校准值集的电机。在原始查询中,我命令计算列(选择前1),我相信这会减慢我的查询速度。

我正在尝试尽可能快地进行此查询。目前执行时间约为0.300秒,我可以通过使用Volume BETWEEN (0.9*1.7006359100341797) AND (1.1*1.7006359100341797)限制表并使用tempTable对计算表(BestMotor)进行排序来将其减少到0.150秒。

我还能做些什么来改善这一点? MotorTable表有大约30,000行。不同的MotorVendors数量约为55。

这是我的原始查询: 我试图为每个不同的MotorVendor找到最接近指定体积的CalX和CalY。

SELECT T.MotorVendor, ((1/(CalX)+1/(CalY))) AS BestMotor FROM (
SELECT MotorVendor,
       (SELECT CalX
        FROM MotorTable AS T2
        WHERE MotorType = 'text' and T2.MotorVendor = Tools.MotorVendor
        ORDER BY abs(Volume - 1.7006359100341797)
        LIMIT 1
       ) AS CalX,
       (SELECT CalY
        FROM MotorTable AS T2
        WHERE MotorType = 'text' and T2.MotorVendor = Tools.MotorVendor
        ORDER BY abs(Volume - 1.7006359100341797)
        LIMIT 1
       ) AS CalY
FROM (SELECT DISTINCT MotorVendor,
      FROM MotorTable) AS Tools) AS T
      WHERE T.CalX != '' AND T.CalY != ''
      ORDER BY BestMotor DESC
      LIMIT 1;

这里是查询使用tempTable对计算列进行排序(在tempTable中对该计算列的索引):

DELETE FROM TempTable;
INSERT INTO TempTable SELECT T.MotorVendor, ((1/(CalX)+1/(CalY))) AS BestMotor FROM (
SELECT MotorVendor,
       (SELECT CalX
        FROM MotorTable AS T2
        WHERE MotorType = 'text' and T2.MotorVendor = Tools.MotorVendor AND Volume BETWEEN (0.9*1.7006359100341797) AND (1.1*1.7006359100341797)
        ORDER BY abs(Volume - 1.7006359100341797)
        LIMIT 1
       ) AS CalX,
       (SELECT CalX
        FROM MotorTable AS T2
        WHERE MotorType = 'text' and T2.MotorVendor = Tools.MotorVendor AND Volume BETWEEN (0.9*1.7006359100341797) AND (1.1*1.7006359100341797)
        ORDER BY abs(Volume - 1.7006359100341797)
        LIMIT 1
       ) AS CalX
FROM (SELECT DISTINCT MotorVendor,
      FROM MotorTable) AS Tools) AS T
      WHERE T.CalX != '' AND T.CalY != ''
      ORDER BY BestMotor DESC
      LIMIT 1;
SELECT MotorVendor, BestMotor FROM TempTable ORDER BY BestMotor DESC LIMIT 1;

... UPDATE

我能把它减少到0.05秒...... 我创建了一个表来存储所有不同的MotorVendor并将主要查询更改为:

SELECT T.MotorVendor FROM (
SELECT MotorVendor,
       (SELECT CalX
        FROM MotorTable AS T2
        WHERE MotorType = 'text' and T2.MotorVendor = Tools.MotorVendor AND Volume BETWEEN (0.9*1.7006359100341797) AND (1.1*1.7006359100341797)
        ORDER BY abs(Volume - 1.7006359100341797)
        LIMIT 1
       ) AS CalX,
       (SELECT CalY
        FROM MotorTable AS T3
        WHERE MotorType = 'text' and T3.MotorVendor = Tools.MotorVendor AND Volume BETWEEN (0.9*1.7006359100341797) AND (1.1*1.7006359100341797)
        ORDER BY abs(Volume - 1.7006359100341797)
        LIMIT 1
        ) AS CalY       
FROM TempMotorVendorTable AS Tools) AS T
      ORDER BY ((1/(CalX)+1/(CalY))) DESC LIMIT 1;

以下是我创建的两个表和索引的模式。

CREATE TABLE MotorTable (
    CalY    real,
    CalX    real,
    Volume  real,
    MotorType   text,
    MotorVendor text
);
CREATE TABLE TempMotorVendorTable (
    MotorVendor TEXT
);
CREATE INDEX `MotorVendorIndex` ON MotorTable (`MotorVendor` ASC)
CREATE INDEX VolumeIndex ON MotorTable (Volume DESC)

EXPLAIN QUERY PLAN的输出

"0" "0" "0" "SCAN TABLE TempMotorVendorTable AS Tools"
"0" "1" "1" "SEARCH TABLE MotorTable USING AUTOMATIC COVERING INDEX (MotorVendor=?)"
"0" "0" "0" "EXECUTE CORRELATED SCALAR SUBQUERY 1"
"1" "0" "0" "SEARCH TABLE MotorTable AS T2 USING INDEX CompToolIndex (MotorVendor=?)"
"1" "0" "0" "USE TEMP B-TREE FOR ORDER BY"
"0" "0" "0" "EXECUTE CORRELATED SCALAR SUBQUERY 2"
"2" "0" "0" "SEARCH TABLE MotorTable AS T3 USING INDEX CompToolIndex (MotorVendor=?)"
"2" "0" "0" "USE TEMP B-TREE FOR ORDER BY"
"0" "0" "0" "USE TEMP B-TREE FOR ORDER BY"

1 个答案:

答案 0 :(得分:1)

ORDER BY abs(Volume - 1.234)无法使用索引进行优化。 (甚至在下一版本的SQLite中也没有表达式索引,因为1.234值不是常量。)

但是,您实际上并不想订购所有行,您只需要最近的一行。 这可以使用不同的查询来完成,该查询会在下面的第一行中搜索该值,并在第一行上面搜索该值(Volume上的这些搜索和排序可以使用索引)。然后取最近的这两行(只排序两行很快):

(SELECT CalX
 FROM (SELECT CalX, diff
       FROM (-- get the largest value at or below 1.234
             SELECT CalX, abs(Volume - 1.234) AS diff
             FROM MotorTable AS T2
             WHERE MotorType = 'text'
               AND MotorVendor = Tools.MotorVendor 
               AND Volume <= 1.234
             ORDER BY Volume DESC
             LIMIT 1)
       UNION ALL
       SELECT CalX, diff
       FROM (-- get the smallest value above 1.234
             SELECT CalX, abs(Volume - 1.234) AS diff
             FROM MotorTable AS T2
             WHERE MotorType = 'text'
               AND MotorVendor = Tools.MotorVendor 
               AND Volume > 1.234
             ORDER BY Volume ASC
             LIMIT 1)
       ORDER BY diff
       LIMIT 1)
)

要使MotorTable中的搜索有效,您需要在所有三列上使用单个索引,并且不等式比较必须在最后一列上:

CREATE INDEX xxx ON MotorTable(MotorType, MotorVendor, Volume);