我有一个应用程序,其中包含一个负责自动构建查询的模块。该模块使普通用户(不是程序员)能够添加额外的"公式"注入查询。该模块似乎不是最优的,因为它翻译了许多简单的公式(例如从同一个" child"表中检索不同的字段)到很多连接中。所以,我决定优化它,以便它可以将类似的公式组合成一个连接。但现在我卡住了,因为我构建了一个几乎小两倍且连接数较少的查询,但运行速度比非优化查询慢四倍。所以,他们在这里: 的未优化:
SELECT f1.id FROM object_73130_ f1
LEFT OUTER JOIN (
SELECT id_field, attr_73400_ FROM (
SELECT m0.id_field, CONCAT_WS(" ",o.attr_73201_,o.attr_73202_,o.attr_73203_) AS attr_73400_ FROM object_73200_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o414_1_0 ON f1.id = o414_1_0.id_field
LEFT OUTER JOIN (
SELECT id_field, attr_73206_ FROM (
SELECT m0.id_field, attr_73206_ FROM object_73200_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o4_1_0 ON f1.id = o4_1_0.id_field
LEFT OUTER JOIN object_73101_ t_4_1_0_0 ON t_4_1_0_0.id = o4_1_0.attr_73206_
LEFT OUTER JOIN (
SELECT id_field, attr_81084_ FROM (
SELECT m0.id_field, attr_81084_ FROM object_73200_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o252_1_0 ON f1.id = o252_1_0.id_field
LEFT OUTER JOIN (
SELECT id_field, attr_73199_ FROM (
SELECT m0.id_field, attr_73199_ FROM object_73195_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o249_1_0 ON f1.id = o249_1_0.id_field
LEFT OUTER JOIN (
SELECT id_field, attr_73217_ FROM (
SELECT m0.id_field, attr_73217_ FROM object_73195_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o250_1_0 ON f1.id = o250_1_0.id_field
LEFT OUTER JOIN (
SELECT id_field, attr_73197_ FROM (
SELECT m0.id_field, attr_73197_ FROM object_73195_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o251_1_0 ON f1.id = o251_1_0.id_field
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1
ORDER BY f1.id_order
) AS o354_1_0 ON (f1.attr_73140_=o354_1_0.id)
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1
ORDER BY f1.id_order
) AS o356_1_0 ON (f1.attr_82225_=o356_1_0.id)
INNER JOIN ( SELECT f1.id FROM object_73130_ f1
WHERE f1.id_obj = 73130 ORDER BY f1.id_order DESC , id DESC LIMIT 0, 300 ) AS lim ON lim.id = f1.id
在我的数据集上运行0.6秒。
这是优化的版本,速度慢了四倍(在同一数据集上):
SELECT f1.id FROM object_73130_ f1
LEFT OUTER JOIN (
SELECT id_field, attr_73206_, attr_81084_ , attr_73400_ FROM (
SELECT m0.id_field, attr_73206_, attr_81084_, CONCAT_WS(" ",o.attr_73201_,o.attr_73202_,o.attr_73203_) AS attr_73400_ FROM object_73200_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o4_1_0 ON f1.id = o4_1_0.id_field
LEFT OUTER JOIN object_73101_ t_4_1_0_0 ON t_4_1_0_0.id = o4_1_0.attr_73206_
LEFT OUTER JOIN (
SELECT id_field, attr_73217_, attr_73197_ , attr_73199_
FROM (
SELECT m0.id_field, attr_73217_, attr_73197_ , attr_73199_
FROM object_73195_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o250_1_0 ON f1.id = o250_1_0.id_field
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1
ORDER BY f1.id_order
) AS o354_1_0 ON (f1.attr_73140_=o354_1_0.id)
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1
ORDER BY f1.id_order
) AS o356_1_0 ON (f1.attr_82225_=o356_1_0.id)
INNER JOIN ( SELECT f1.id FROM object_73130_ f1 WHERE f1.id_obj = 73130 ORDER BY f1.id_order DESC, id DESC LIMIT 0, 300 )
AS lim ON lim.id = f1.id
查询本身可能对您没有意义。但事实是,第二个优化查询的运行速度要慢得多。我尝试调查原因,发现由于attr_73197_
类型的一个字段VARCHAR(128)
,第二个查询变慢了。在第一个non-optimized
查询中,此字段转换为单个联接:
LEFT OUTER JOIN (
SELECT id_field, attr_73197_ FROM (
SELECT m0.id_field, attr_73197_ FROM object_73195_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o251_1_0 ON f1.id = o251_1_0.id_field
在优化版本中,此字段在此处注入另一个连接:
LEFT OUTER JOIN (
SELECT id_field, attr_73217_, attr_73197_ , attr_73199_
FROM (
SELECT m0.id_field, attr_73217_, attr_73197_ , attr_73199_
FROM object_73195_ o
INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id
ORDER BY o.id_order
) AS o GROUP BY id_field
) AS o250_1_0 ON f1.id = o250_1_0.id_field
但这一切导致速度非常低。如果我注释掉这个字段,那么优化版本现在开始比其对应版本更快地运行。那怎么可能呢?
修改
我现在开始认为问题可能来自TEXT
类型的字段。当这个字段单独出现时,MySQL可能会进行一些内部优化,但是当它带有其他字段时,它不会这样做。我不知道。
修改
优化查询的EXPLAIN
命令的结果:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived8> ALL \N \N \N \N 300
1 PRIMARY f1 eq_ref PRIMARY PRIMARY 8 lim.id 1
1 PRIMARY <derived2> ALL \N \N \N \N 949
1 PRIMARY t_4_1_0_0 eq_ref PRIMARY PRIMARY 8 o4_1_0.attr_73206_ 1 Using index
1 PRIMARY <derived4> ALL \N \N \N \N 947
1 PRIMARY <derived6> ALL \N \N \N \N 13
1 PRIMARY <derived7> ALL \N \N \N \N 13
8 DERIVED f1 ALL \N \N \N \N 947 Using filesort
7 DERIVED f1 ALL \N \N \N \N 13 Using filesort
6 DERIVED f1 ALL \N \N \N \N 13 Using filesort
4 DERIVED <derived5> ALL \N \N \N \N 947 Using temporary; Using filesort
5 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
5 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
2 DERIVED <derived3> ALL \N \N \N \N 949 Using temporary; Using filesort
3 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
3 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
修改
非优化查询的EXPLAIN
命令的结果:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived16> ALL \N \N \N \N 300
1 PRIMARY f1 eq_ref PRIMARY PRIMARY 8 lim.id 1
1 PRIMARY <derived2> ALL \N \N \N \N 949
1 PRIMARY <derived4> ALL \N \N \N \N 949
1 PRIMARY t_4_1_0_0 eq_ref PRIMARY PRIMARY 8 o4_1_0.attr_73206_ 1 Using index
1 PRIMARY <derived6> ALL \N \N \N \N 949
1 PRIMARY <derived8> ALL \N \N \N \N 947
1 PRIMARY <derived10> ALL \N \N \N \N 947
1 PRIMARY <derived12> ALL \N \N \N \N 947
1 PRIMARY <derived14> ALL \N \N \N \N 13
1 PRIMARY <derived15> ALL \N \N \N \N 13
16 DERIVED f1 ALL \N \N \N \N 947 Using filesort
15 DERIVED f1 ALL \N \N \N \N 13 Using filesort
14 DERIVED f1 ALL \N \N \N \N 13 Using filesort
12 DERIVED <derived13> ALL \N \N \N \N 947 Using temporary; Using filesort
13 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
13 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
10 DERIVED <derived11> ALL \N \N \N \N 947 Using temporary; Using filesort
11 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
11 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
8 DERIVED <derived9> ALL \N \N \N \N 947 Using temporary; Using filesort
9 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
9 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
6 DERIVED <derived7> ALL \N \N \N \N 949 Using temporary; Using filesort
7 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
7 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
4 DERIVED <derived5> ALL \N \N \N \N 949 Using temporary; Using filesort
5 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
5 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
2 DERIVED <derived3> ALL \N \N \N \N 949 Using temporary; Using filesort
3 DERIVED m0 ALL indx_master_slave \N \N \N 2782 Using where; Using temporary; Using filesort
3 DERIVED o eq_ref PRIMARY PRIMARY 8 project.m0.id_slave_field 1 Using where
修改
我不知道EXPLAIN
如何解释我的奇怪案例。但我的最终结论是,这完全是因为TEXT
类型的领域。由于一些神秘的原因,它在一个join
中表现良好,但它在join
与其他一些领域中造成了很多麻烦。但我希望有更严格的解释。