需要帮助来改进MYSQL SubQuery性能

时间:2017-07-07 05:04:38

标签: mysql sql query-performance

我刚刚学习MYSQL,我有这样的MySql子查询:

EXPLAIN EXTENDED SELECT brand_name, stars, hh_stock, hh_stock_value, sales_monthly_1, sales_monthly_2, sales_monthly_3, sold_monthly_1, sold_monthly_2,
  sold_monthly_3, price_uvp, price_ecp, price_default, price_margin AS margin, vc_percent as vc, cogs, products_length, products_id, material_expenses,
  MAX(price) AS products_price, SUM(total_sales) AS total_sales,
  IFNULL(MAX(active_age), DATEDIFF(NOW(), products_date_added)) AS products_age, DATEDIFF(NOW(), products_date_added) AS jng_products_age,
  AVG(sales_weekly) AS sales_weekly, AVG(sales_monthly) AS sales_monthly, SUM(total_sold) AS total_sold, SUM(total_returned) AS total_returned,
  ((SUM(total_returned)/SUM(total_sold)) * 100) AS returned_rate
FROM
  (
    SELECT p.products_id, jc.price, jc.price_end_customer AS price_ecp, jc.total_sales, jc.active_age, jc.sales_weekly,
      jc.sales_monthly, jc.total_sold, jc.total_returned, jc.price_uvp, p.price_margin, p.vc_percent, p.material_expenses,
      p.products_date_added, p.stars , pb.brand_name, p.family_id, p.products_price_default AS price_default, pl.sales_monthly_1,
      pl.sales_monthly_2, pl.sales_monthly_3, pl.sold_monthly_1, pl.sold_monthly_2, pl.sold_monthly_3, pst.stock AS hh_stock,
      (pst.stock * p.average_stock_value) AS hh_stock_value, pnc.products_length,
      IF(ploc.cogs IS NULL OR ploc.cogs=0,
         (CASE p.complexity
          WHEN 'F' THEN ROUND(5*(p.material_expenses+(7.5/100*p.material_expenses)+1.7+0.25+2.2)/100+(p.material_expenses+(7.5/100*p.material_expenses)+1.7+0.25+2.2),2)
          WHEN 'E' THEN ROUND(5*(p.material_expenses+(7.5/100*p.material_expenses)+1.7+0.25+2.2)/100+(p.material_expenses+(7.5/100*p.material_expenses)+1.7+0.25+2.2),2)
          WHEN 'N' THEN ROUND(5*(p.material_expenses+(7.5/100*p.material_expenses)+2.4+0.25+2.2)/100+(p.material_expenses+(7.5/100*p.material_expenses)+2.4+0.25+2.2),2)
          WHEN 'M' THEN ROUND(5*(p.material_expenses+(7.5/100*p.material_expenses)+2.4+0.25+2.2)/100+(p.material_expenses+(7.5/100*p.material_expenses)+2.4+0.25+2.2),2)
          WHEN 'I' THEN ROUND(5*(p.material_expenses+(7.5/100*p.material_expenses)+3.5+0.25+2.2)/100+(p.material_expenses+(7.5/100*p.material_expenses)+3.5+0.25+2.2),2)
          WHEN 'H' THEN ROUND(5*(p.material_expenses+(7.5/100*p.material_expenses)+3.5+0.25+2.2)/100+(p.material_expenses+(7.5/100*p.material_expenses)+3.5+0.25+2.2),2)
          ELSE ROUND(5*(p.material_expenses+(7.5/100*p.material_expenses)+5+0.25+2.2)/100+(p.material_expenses+(7.5/100*p.material_expenses)+5+0.25+2.2),2) END), ploc.cogs) AS cogs
    FROM products p
      LEFT  JOIN jng_sp_catalog jc ON  jc.products_id=p.products_id
      LEFT JOIN products_description pd ON pd.products_id = p.products_id AND pd.language_id = 2
      LEFT JOIN products_description2 pd2 ON pd2.products_id = p.products_id
      LEFT JOIN products_brand pb ON pb.products_brand_id = p.products_brand_id
      LEFT JOIN products_log pl ON pl.products_id = p.products_id
      LEFT JOIN products_log_static pls ON pls.products_id=p.products_id
      LEFT JOIN products_local ploc ON ploc.products_id = p.products_id
      LEFT JOIN products_non_configurator pnc ON pnc.products_id = p.products_id
      INNER JOIN
      (
        SELECT shp.products_id, CONCAT(',', GROUP_CONCAT(shp.styles_id), ',') AS styles_id
        FROM styles_has_products shp GROUP BY shp.products_id HAVING styles_id NOT LIKE '%,1967,%') subquery_styles ON subquery_styles.products_id = p.products_id
      LEFT JOIN products_stock_temp pst ON pst.products_id=p.products_id WHERE p.active_status='1'  AND p.categories_top_id  =  '1') dt  GROUP BY products_id ORDER BY  products_id;

解释的结果是这样的:

+----+-------------+------------+------------+--------+---------------------+-------------+---------+------------------------------------+--------+----------+----------------------------------------------+
| id | select_type | table      | partitions | type   | possible_keys       | key         | key_len | ref                                | rows   | filtered | Extra                                        |
+----+-------------+------------+------------+--------+---------------------+-------------+---------+------------------------------------+--------+----------+----------------------------------------------+
|  1 | PRIMARY     | p          | NULL       | ALL    | PRIMARY             | NULL        | NULL    | NULL                               |  40458 |     1.00 | Using where; Using temporary; Using filesort |
|  1 | PRIMARY     | pb         | NULL       | eq_ref | PRIMARY             | PRIMARY     | 4       | manobo_central.p.products_brand_id |      1 |   100.00 | NULL                                         |
|  1 | PRIMARY     | ploc       | NULL       | eq_ref | PRIMARY             | PRIMARY     | 4       | manobo_central.p.products_id       |      1 |   100.00 | NULL                                         |
|  1 | PRIMARY     | pl         | NULL       | eq_ref | PRIMARY             | PRIMARY     | 4       | manobo_central.p.products_id       |      1 |   100.00 | Using where                                  |
|  1 | PRIMARY     | pls        | NULL       | eq_ref | PRIMARY             | PRIMARY     | 4       | manobo_central.p.products_id       |      1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | pst        | NULL       | eq_ref | PRIMARY             | PRIMARY     | 4       | manobo_central.p.products_id       |      1 |   100.00 | NULL                                         |
|  1 | PRIMARY     | pd2        | NULL       | eq_ref | PRIMARY             | PRIMARY     | 4       | manobo_central.p.products_id       |      1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | pnc        | NULL       | eq_ref | PRIMARY             | PRIMARY     | 4       | manobo_central.p.products_id       |      1 |   100.00 | Using where                                  |
|  1 | PRIMARY     | pd         | NULL       | eq_ref | PRIMARY             | PRIMARY     | 8       | manobo_central.p.products_id,const |      1 |   100.00 | Using index                                  |
|  1 | PRIMARY     | jc         | NULL       | ref    | products_id         | products_id | 4       | manobo_central.p.products_id       |      4 |   100.00 | Using where                                  |
|  1 | PRIMARY     | <derived3> | NULL       | ref    | <auto_key0>         | <auto_key0> | 4       | manobo_central.p.products_id       |     10 |   100.00 | Using where                                  |
|  3 | DERIVED     | shp        | NULL       | index  | PRIMARY,products_id | PRIMARY     | 8       | NULL                               | 208226 |   100.00 | Using index; Using filesort                  |
+----+-------------+------------+------------+--------+---------------------+-------------+---------+------------------------------------+--------+----------+----------------------------------------------+

我有选择。

  1. 我将删除子查询并使用VIEWS输出数据,就像使用查询一样。因为我在FROM中有子查询,所以我将使用VIEWS中的VIEWS。但有些人表示会影响表演。你们怎么想这个?
  2. 我仍然会使用子查询,但会尝试搜索如何优化查询。对于这个,我想问你们,对于EXPLAIN TABLE中的第一个结果行,它显示了表格生成p,类型&#39;所有&#39;,如何避免&#39; all&#39; ?我设法使用了类型&#39; eq_ref&#39;对于其他表,但仍然不知道为什么产品表是全部&#39;?
  3. 再次 你觉得我需要切换到VIEW吗?或者只是尝试再次优化子查询。

    非常感谢!

    编辑:表产品索引

    create index family_id on products (family_id);
    create index idx_products_date_added on products (products_date_added);
    create index material_expenses on products (material_expenses);
    create index products_brand_id on products (products_brand_id);
    create index products_ean on products (products_ean);
    create index products_status on products (products_status);
    create index tb_status on products (tb_status);
    

    编辑:table style_has_products

    CREATE TABLE `styles_has_products` (
      `styles_id` int(10) unsigned NOT NULL DEFAULT '0',
      `products_id` int(10) unsigned NOT NULL DEFAULT '0',
      `date_added` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
      PRIMARY KEY (`styles_id`,`products_id`),
      KEY `products_id` (`products_id`)
    ) ENGINE=MyISAM DEFAULT CHARSET=latin1
    

3 个答案:

答案 0 :(得分:1)

首先,从不编写如此复杂的查询以供实时使用。我建议进行批处理和维护数据仓库。并在数据仓库上使用实时查询。

仍然有许多事情你不应该在实时使用SQL查询来获得性能。就像从不使用更多的连接操作,永远不要放更多if else条件,永远不要应用group,特别是如果表很大,在表中寻找合适的索引,分区结构。

答案 1 :(得分:0)

我注意到的第一件事是你的subquery_styles。除了过滤之外,您不使用其结果。但是,在我看来,标准属于WHERE条款。由于您似乎要排除存在style_id 1967年的产品,我会使用NOT EXISTSNOT IN

WHERE p.active_status = 1
AND p.categories_top_id = 1
AND p.products_id NOT IN
(
  SELECT products_id
  FROM styles_has_products
  WHERE styles_id = 1967
)

第二件事是你的查询没有合适的索引。您正在选择具有active_status 1和categories_top_id 1的产品,但这些列上没有索引。由于product_id上的第三个条件不匹配style_id 1967,我建议使用以下索引之一:

create index idx1 on products (active_status, categories_top_id, products_id);
create index idx2 on products (categories_top_id, active_status, products_id);

创建两者,查看正在使用哪一个,然后删除另一个。

可以而且可能应该优化/更改的最后一点是您的聚合。但为了在这里提供帮助,我必须知道该表的唯一键。一旦你发布它们,我会延伸这个答案: - )

答案 2 :(得分:0)

根据Thorsten的建议,而不是NOT IN ( SELECT ),使用

NOT EXISTS( SELECT * FROM styles_has_products
       WHERE products_id = p.products_id
         AND styles_id = 1967 )

styles_has_products以任意顺序需要INDEX(products_id, styles_id)

请告诉我们SHOW CREATE TABLE styles_has_products。如果是多个:多个映射表,请参阅提示here

索引需要在进入的桌面上,而不是来自的。因此,products的索引列表可能不会被使用。此复合索引可能非常有用:

INDEX(categories_top_id, active_status) -- in either order

VIEWs只是语法糖;它们本身并不提供任何性能优势。在某些情况下,他们会损害表现。

不使用

pdpd2pls和其他一些内容;删除他们的JOINs

SUMsAVGs可能不正确。这是因为&#34;爆炸 - 内爆&#34;与JOIN + GROUP BY一起发生。清理其他一些内容,然后我们可以讨论如何重新排列内容,以便SUMsAVGsproduct_id只执行一行。