SQL优化难题:更大的查询+更多的连接=更高的速度。为什么?

时间:2014-03-12 22:49:14

标签: mysql sql optimization

我有一个应用程序,其中包含一个负责自动构建查询的模块。该模块使普通用户(不是程序员)能够添加额外的"公式"注入查询。该模块似乎不是最优的,因为它翻译了许多简单的公式(例如从同一个" child"表中检索不同的字段)到很多连接中。所以,我决定优化它,以便它可以将类似的公式组合成一个连接。但现在我卡住了,因为我构建了一个几乎小两倍且连接数较少的查询,但运行速度比非优化查询慢四倍。所以,他们在这里: 的未优化

SELECT f1.id FROM object_73130_ f1   
LEFT OUTER JOIN (
SELECT id_field, attr_73400_ FROM (
    SELECT m0.id_field, CONCAT_WS(" ",o.attr_73201_,o.attr_73202_,o.attr_73203_) AS attr_73400_ FROM object_73200_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o414_1_0 ON f1.id = o414_1_0.id_field    
LEFT OUTER JOIN (
SELECT id_field, attr_73206_ FROM (
    SELECT m0.id_field,  attr_73206_  FROM object_73200_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o4_1_0 ON f1.id = o4_1_0.id_field  
LEFT OUTER JOIN object_73101_ t_4_1_0_0 ON t_4_1_0_0.id = o4_1_0.attr_73206_    
LEFT OUTER JOIN (
SELECT id_field, attr_81084_ FROM (
    SELECT m0.id_field,  attr_81084_  FROM object_73200_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o252_1_0 ON f1.id = o252_1_0.id_field   
LEFT OUTER JOIN (
SELECT id_field, attr_73199_ FROM (
    SELECT m0.id_field,  attr_73199_  FROM object_73195_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o249_1_0 ON f1.id = o249_1_0.id_field    
LEFT OUTER JOIN (
SELECT id_field, attr_73217_ FROM (
    SELECT m0.id_field,  attr_73217_  FROM object_73195_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o250_1_0 ON f1.id = o250_1_0.id_field    
LEFT OUTER JOIN (
SELECT id_field, attr_73197_ FROM (
    SELECT m0.id_field,  attr_73197_  FROM object_73195_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o251_1_0 ON f1.id = o251_1_0.id_field    
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1 
ORDER BY  f1.id_order  
) AS o354_1_0 ON (f1.attr_73140_=o354_1_0.id)    
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1 
ORDER BY  f1.id_order  
) AS o356_1_0 ON (f1.attr_82225_=o356_1_0.id)    
INNER JOIN ( SELECT f1.id FROM object_73130_ f1   
WHERE f1.id_obj = 73130 ORDER BY  f1.id_order  DESC , id DESC LIMIT 0, 300 ) AS lim ON lim.id = f1.id 

在我的数据集上运行0.6秒。

这是优化的版本,速度慢了四倍(在同一数据集上):

SELECT f1.id FROM object_73130_ f1  
LEFT OUTER JOIN (
SELECT id_field, attr_73206_, attr_81084_ , attr_73400_   FROM (
    SELECT m0.id_field,  attr_73206_, attr_81084_, CONCAT_WS(" ",o.attr_73201_,o.attr_73202_,o.attr_73203_) AS attr_73400_ FROM object_73200_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73200 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o4_1_0 ON f1.id = o4_1_0.id_field  
LEFT OUTER JOIN object_73101_ t_4_1_0_0 ON t_4_1_0_0.id = o4_1_0.attr_73206_       
LEFT OUTER JOIN (
SELECT id_field, attr_73217_, attr_73197_ , attr_73199_  
FROM (
    SELECT m0.id_field,  attr_73217_, attr_73197_ , attr_73199_   
    FROM object_73195_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o250_1_0 ON f1.id = o250_1_0.id_field   
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1
ORDER BY  f1.id_order  
) AS o354_1_0 ON (f1.attr_73140_=o354_1_0.id)    
LEFT OUTER JOIN (
SELECT attr_82212_ ,id FROM object_69534_ f1 
ORDER BY  f1.id_order  
) AS o356_1_0 ON (f1.attr_82225_=o356_1_0.id)    
INNER JOIN ( SELECT f1.id FROM object_73130_ f1   WHERE f1.id_obj = 73130 ORDER BY  f1.id_order  DESC, id DESC LIMIT 0, 300 ) 
AS lim ON lim.id = f1.id

查询本身可能对您没有意义。但事实是,第二个优化查询的运行速度要慢得多。我尝试调查原因,发现由于attr_73197_类型的一个字段VARCHAR(128),第二个查询变慢了。在第一个non-optimized查询中,此字段转换为单个联接:

LEFT OUTER JOIN (
SELECT id_field, attr_73197_ FROM (
    SELECT m0.id_field,  attr_73197_  FROM object_73195_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
)  AS o251_1_0 ON f1.id = o251_1_0.id_field 

在优化版本中,此字段在此处注入另一个连接:

LEFT OUTER JOIN (
SELECT id_field, attr_73217_, attr_73197_ , attr_73199_  
FROM (
    SELECT m0.id_field,  attr_73217_, attr_73197_ , attr_73199_   
    FROM object_73195_ o
    INNER JOIN master_slave m0 ON m0.id_object = 73130 AND m0.id_master = 73195 AND m0.id_slave_field = o.id 
    ORDER BY  o.id_order 
) AS o  GROUP BY id_field 
 )  AS o250_1_0 ON f1.id = o250_1_0.id_field

但这一切导致速度非常低。如果我注释掉这个字段,那么优化版本现在开始比其对应版本更快地运行。那怎么可能呢?

修改

我现在开始认为问题可能来自TEXT类型的字段。当这个字段单独出现时,MySQL可能会进行一些内部优化,但是当它带有其他字段时,它不会这样做。我不知道。

修改 优化查询的EXPLAIN命令的结果:

id  select_type table       type    possible_keys       key         key_len ref                 rows    Extra
1   PRIMARY     <derived8>  ALL     \N                  \N          \N      \N                  300 
1   PRIMARY     f1  eq_ref  PRIMARY PRIMARY             8           lim.id  1   
1   PRIMARY     <derived2>  ALL     \N                  \N          \N      \N                  949 
1   PRIMARY     t_4_1_0_0   eq_ref  PRIMARY             PRIMARY     8       o4_1_0.attr_73206_  1       Using index
1   PRIMARY     <derived4>  ALL     \N                  \N          \N      \N                  947 
1   PRIMARY     <derived6>  ALL     \N                  \N          \N      \N                  13  
1   PRIMARY     <derived7>  ALL     \N                  \N          \N      \N                  13  
8   DERIVED     f1          ALL     \N   \N     \N                  \N                          947                                 Using filesort
7   DERIVED     f1  ALL \N  \N      \N                  \N                                     13                                   Using filesort
6   DERIVED     f1  ALL \N  \N      \N                  \N                                     13                                   Using filesort
4   DERIVED     <derived5>  ALL     \N                  \N          \N      \N                  947     Using temporary; Using filesort
5   DERIVED     m0          ALL     indx_master_slave   \N          \N      \N                  2782    Using where; Using temporary; Using filesort
5   DERIVED     o           eq_ref  PRIMARY             PRIMARY     8       project.m0.id_slave_field   1   Using where
2   DERIVED     <derived3>  ALL     \N                  \N          \N      \N                  949 Using temporary; Using filesort
3   DERIVED     m0          ALL     indx_master_slave   \N          \N      \N                  2782    Using where; Using temporary; Using filesort
3   DERIVED     o           eq_ref  PRIMARY             PRIMARY     8       project.m0.id_slave_field   1   Using where

修改 非优化查询的EXPLAIN命令的结果:

id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   PRIMARY <derived16> ALL \N  \N  \N  \N  300 
1   PRIMARY f1          eq_ref  PRIMARY PRIMARY 8   lim.id  1   
1   PRIMARY <derived2>  ALL     \N  \N  \N  \N  949 
1   PRIMARY <derived4>  ALL     \N  \N  \N  \N  949 
1   PRIMARY t_4_1_0_0   eq_ref  PRIMARY PRIMARY 8   o4_1_0.attr_73206_       1  Using index
1   PRIMARY <derived6>  ALL     \N  \N  \N  \N  949 
1   PRIMARY <derived8>  ALL     \N  \N  \N  \N  947 
1   PRIMARY <derived10> ALL     \N  \N  \N  \N  947 
1   PRIMARY <derived12> ALL     \N  \N  \N  \N  947 
1   PRIMARY <derived14> ALL     \N  \N  \N  \N  13  
1   PRIMARY <derived15> ALL     \N  \N  \N  \N  13  
16  DERIVED f1          ALL     \N  \N  \N  \N  947 Using filesort
15  DERIVED f1          ALL     \N  \N  \N  \N  13  Using filesort
14  DERIVED f1          ALL     \N  \N  \N  \N  13  Using filesort
12  DERIVED <derived13> ALL     \N  \N  \N  \N  947 Using temporary; Using filesort
13  DERIVED m0          ALL     indx_master_slave   \N  \N  \N  2782    Using where; Using temporary; Using filesort
13  DERIVED o           eq_ref  PRIMARY PRIMARY 8   project.m0.id_slave_field   1   Using where
10  DERIVED <derived11> ALL     \N  \N  \N  \N  947 Using temporary; Using filesort
11  DERIVED m0          ALL     indx_master_slave   \N  \N  \N  2782    Using where; Using temporary; Using filesort
11  DERIVED o           eq_ref  PRIMARY PRIMARY 8   project.m0.id_slave_field   1   Using where
8   DERIVED <derived9>  ALL     \N  \N  \N  \N  947 Using temporary; Using filesort
9   DERIVED m0          ALL     indx_master_slave   \N  \N  \N  2782    Using where; Using temporary; Using filesort
9   DERIVED o           eq_ref  PRIMARY PRIMARY 8   project.m0.id_slave_field   1   Using where
6   DERIVED <derived7>  ALL     \N  \N  \N  \N  949 Using temporary; Using filesort
7   DERIVED m0          ALL     indx_master_slave   \N  \N  \N  2782    Using where; Using temporary; Using filesort
7   DERIVED o           eq_ref  PRIMARY PRIMARY 8   project.m0.id_slave_field   1   Using where
4   DERIVED <derived5>  ALL     \N  \N  \N  \N  949 Using temporary; Using filesort
5   DERIVED m0          ALL     indx_master_slave   \N  \N  \N  2782    Using where; Using temporary; Using filesort
5   DERIVED o           eq_ref  PRIMARY PRIMARY 8   project.m0.id_slave_field   1   Using where
2   DERIVED <derived3>  ALL     \N  \N  \N  \N  949 Using temporary; Using filesort
3   DERIVED m0          ALL     indx_master_slave   \N  \N  \N  2782    Using where; Using temporary; Using filesort
3   DERIVED o           eq_ref  PRIMARY PRIMARY 8   project.m0.id_slave_field   1   Using where

修改 我不知道EXPLAIN如何解释我的奇怪案例。但我的最终结论是,这完全是因为TEXT类型的领域。由于一些神秘的原因,它在一个join中表现良好,但它在join与其他一些领域中造成了很多麻烦。但我希望有更严格的解释。

0 个答案:

没有答案