MySQL慢查询和memcached

时间:2012-04-24 13:31:24

标签: mysql performance memcached

我的缓存查询缓存了memcached,它很好,但有时我们在mysql中遇到了很多慢查询,好像memcached停止了然后我们的网站停止了。

慢查询图表:http://i.imgur.com/ReyWe.png

此时我们在30秒内完成了大约100次慢速查询。它是什么? 我们的询问:

# Query_time: 5.942602  Lock_time: 0.010214 Rows_sent: 10000  Rows_examined: 493139  

SET timestamp=1335194149;  
SELECT story_id FROM dug_stories d WHERE d.story_is_permanent=0  AND  
                       (  
                        (d.story_time>1335190543 AND (d.story_pluses-d.story_minuses) > 5)  
                        OR  
                        ( ( (d.story_pluses-d.story_minuses) > 12 OR (d.story_pluses >= 10 AND d.story_minuses<=0) )  AND d.story_cat!=48 AND d.st  
ory_cat!=131 AND d.story_cat!=55 AND d.story_cat!=44 AND d.story_cat!=126 AND d.story_cat!=53 AND d.story_cat!=370 AND d.story_cat!=381 AND d.stor  
y_cat!=304 AND d.story_cat!=497)  OR (d.story_cat=48 AND (d.story_pluses-d.story_minuses) > 9) OR (d.story_cat=131 AND (d.story_pluses-d.story_min  
uses) > 8) OR (d.story_cat=55 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=44 AND (d.story_pluses-d.story_minuses) > 9) OR (d.story_c  
at=126 AND (d.story_pluses-d.story_minuses) > 13) OR (d.story_cat=53 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=370 AND (d.story_pl  
uses-d.story_minuses) > 8) OR (d.story_cat=381 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=304 AND (d.story_pluses-d.story_minuses)  
> 8) OR (d.story_cat=497 AND (d.story_pluses-d.story_minuses) > 9)  
                       )  
                       ORDER BY d.story_rating DESC, d.story_time DESC LIMIT 0, 10000;  



稍微调整后的查询(而不是所有!=类别,我更改为NOT Category IN(要排除的类别列表),但格式化以便于阅读...另外,请注意,SET TIMESTAMP变量不是甚至在查询中使用...它是一个硬编码的值。

SET timestamp=1335194149;  
SELECT 
      story_id 
   FROM 
      dug_stories d 
   WHERE 
          d.story_is_permanent = 0  
      AND (     ( d.story_time > 1335190543 AND d.story_pluses - d.story_minuses > 5 )  
             OR (    (    d.story_pluses - d.story_minuses > 12 
                       OR (   d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
                     )  
                  AND NOT d.story_cat IN ( 44, 48, 53, 55, 126, 131, 304, 370, 381, 497 )
                )
             OR ( d.story_cat = 44  AND d.story_pluses - d.story_minuses > 9 ) 
             OR ( d.story_cat = 48  AND d.story_pluses - d.story_minuses > 9 ) 
             OR ( d.story_cat = 53  AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 55  AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 126 AND d.story_pluses - d.story_minuses > 13 ) 
             OR ( d.story_cat = 131 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 304 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 370 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 381 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 497 AND d.story_pluses - d.story_minuses > 9 )  
          )  
   ORDER BY 
      d.story_rating DESC, 
      d.story_time DESC 
   LIMIT 
      0, 10000;  

非常感谢您的帮助! 抱歉我的英语很差:)。

2 个答案:

答案 0 :(得分:0)

如果rows_examined大于结果集,则慢速日志可能表示索引未被充分利用。

理想情况下,MySQL应该能够为WHERE子句中的所有过滤表达式使用单个索引。由于所有嵌套的AND和OR以及对多个列的引用,您的WHERE子句非常复杂。 MySQL可能无法使用索引来优化此查询,即使您有一个。

通常,这种类型的查询可以通过将其分成几个较小的查询来改进,每个查询都能够使用索引,并使用UNION将结果组合到单个查询中。

例如,如果我在first_name上有索引,在last_name上有索引,则以下查询:

SELECT *
FROM Users
WHERE active = true
  AND ((first_name = 'Marcus') OR (last_name = 'Adams'))

可以这样改进:

SELECT *
FROM Users
WHERE active = true
  AND first_name = 'Marcus'
UNION
SELECT *
FROM Users
WHERE active = true
  AND last_name = 'Adams'

在LIMIT子句中检查的行数多于10000,这也表明ORDER BY子句可能正在使用文件排序,而不是按顺序返回行的查询。这使得MySQL可以完成所有工作,即使您只需要一部分记录。您很可能需要使用覆盖索引来处理所有条件,这可能是不可能的,或者您需要删除ORDER BY子句,或者,如您所示,依靠缓存来节省时间。

查询时间是高负载服务器上的性能测量不佳,因为任何查询在繁忙的服务器上都可能很慢。您应该使用EXPLAINSHOW PROFILE来分析您的查询并对其进行改进,以免他们对您的服务器施加太多负担。

允许在服务器上运行太多线程会导致级联效应,从而导致服务器停止运行。如果您使用InnoDB,请确保thread concurrency不是太高。最好让每个人一次等待并服务8个线程(非常快),而不是一次尝试服务100个或无限制的线程(非常慢)。

答案 1 :(得分:0)

除了调整查询的可读性之外,我还会尝试重新构建查询 我会确保你有一个索引(Story_Is_Permanent,Story_Cat,Story_Time)

我假设你有一个“故事类别”表,其中包含文字描述,以便向用户展示。由于您无论如何都要查询所有类别,但仅基于某些类别,您要么包含或排除...如果包含,则仅基于此类别的PLUSES和MINUSES的净计数。我会将列添加到该类别表中......类似于

DefaultToInclude   integer as 1 or 0
PlusMinusNetCount  integer

然后,加入类别表并简化查询,例如

SELECT 
      story_id 
   FROM 
      dug_stories d
         JOIN YourCategories YC
            on d.story_cat = YC.categoryID
   WHERE 
          d.story_is_permanent = 0  
      AND (     ( d.story_time > 1335190543 AND d.story_pluses - d.story_minuses > 5 )  
             OR (     YC.DefaultToInclude = 0 
                  AND (    d.story_pluses - d.story_minuses > 12 
                       OR (d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
                      )  
                )
             OR (     YC.DefaultToInclude = 1 
                  AND d.story_pluses-d.story_minuses > YC.PlusMinusNetCount )
          )
 ORDER BY 
      d.story_rating DESC, 
      d.story_time DESC 
   LIMIT 
      0, 10000;  




<强>的

(     YC.DefaultToInclude = 0 
  AND (    d.story_pluses - d.story_minuses > 12 
        OR (d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
      )  
)

最终会取代

OR (    (    d.story_pluses - d.story_minuses > 12 
          OR (   d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
        )  
    AND d.story_cat!=48 
    AND d.story_cat!=131 
    AND d.story_cat!=55 
    AND d.story_cat!=44 
    AND d.story_cat!=126 
    AND d.story_cat!=53 
    AND d.story_cat!=370 
    AND d.story_cat!=381 
    AND d.story_cat!=304 
    AND d.story_cat!=497 
   )

<强>的

 OR (     YC.DefaultToInclude = 1 
      AND d.story_pluses-d.story_minuses > YC.PlusMinusNetCount )

最终会取代

( OR ( d.story_cat = 44  AND d.story_pluses - d.story_minuses > 9 ) 
  OR ( d.story_cat = 48  AND d.story_pluses - d.story_minuses > 9 ) 
  OR ( d.story_cat = 53  AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 55  AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 126 AND d.story_pluses - d.story_minuses > 13 ) 
  OR ( d.story_cat = 131 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 304 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 370 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 381 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 497 AND d.story_pluses - d.story_minuses > 9 ) 
) 

这样可以灵活地简单地包含/排除具有相应“净”计数的另一个类别,而无需oooopps为您已经很长的查询添加更多OR / AND条件。没有什么,似乎它总是必须通过整个“dug_Stories”表,因为你正在查看一个条件的每个类别和另一个条件的每个其他类别。您的时间戳仅适用于NOT EQUAL类别,但您的其他条件没有任何时间戳限制(除非您的查询中出现错误)。