MySQL查询缓慢或卡在发送数据上

时间:2015-10-16 13:15:06

标签: mysql sql indexing query-optimization

我在MySQL 5.5.37中有以下表模式,该表到目前为止已经有1685412行并且还在增长。

CREATE TABLE `events` (
  `ID` int(11) NOT NULL AUTO_INCREMENT,
  `tSource` varchar(255) NOT NULL,
  `tStatus` varchar(6) NOT NULL,
  `tRef` varchar(255) DEFAULT NULL,
  `externalEventID` varchar(255) DEFAULT NULL,
  `eName` varchar(255) DEFAULT NULL,
  `eDescription` mediumtext,
  `eStatus` varchar(45) DEFAULT 'OK',
  `ePri` tinyint(6) NOT NULL DEFAULT '5',
  `eSta` tinyint(1) NOT NULL DEFAULT '0',
  `dId` varchar(45) DEFAULT NULL,
  `dName` varchar(255) DEFAULT NULL,
  `dIp` varchar(15) DEFAULT NULL,
  `customerName` varchar(255) DEFAULT NULL,
  `dateCreated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `dateUpdated` datetime NOT NULL,
  `externaltRef` varchar(255) DEFAULT NULL,
  `eventNotes` text,
  `bId` varchar(45) DEFAULT NULL,
  `supp` tinyint(1) NOT NULL DEFAULT '0',
  `processed` tinyint(4) NOT NULL DEFAULT '0',
  `rString` text,
  `rUrl` varchar(255) DEFAULT NULL,
  `rEsc` varchar(255) DEFAULT NULL,
  `rEscO` varchar(255) DEFAULT NULL,
  `eCle` tinyint(4) NOT NULL DEFAULT '0',
  PRIMARY KEY (`ID`),
  KEY `ePri_count` (`supp`,`eStatus`,`dateCreated`,`tStatus`),
  KEY `event_device_date_status` (`eName`,`dName`,`dateUpdated`,`tStatus`),
  KEY `status_customer_date_tstatus` (`eStatus`,`customerName`,`dateCreated`,`tStatus`),
  KEY `ePri_count_top` (`supp`,`eStatus`,`processed`,`dateCreated`,`customerName`,`tStatus`),
  KEY `event_suppress` (`dateCreated`,`processed`,`supp`),
  KEY `related_events` (`dName`,`tRef`,`tStatus`,`processed`,`dateUpdated`),
  KEY `eName_eStatus_dName_date_proc` (`processed`,`eStatus`,`dName`,`dateCreated`,`eName`),
  KEY `proc_tStatus_customer_date_priority` (`processed`,`tStatus`,`customerName`,`dateCreated`,`ePri`)
) ENGINE=InnoDB AUTO_INCREMENT=1685400 DEFAULT CHARSET=utf8

EXPLAIN 
SELECT ID AS id
     , tSource
     , eDescription
     , eName
     , dId
     , dIp
     , tRef
     , customerName
     , dName
     , ePri
     , dateCreated
     , tStatus
     , supp 
  FROM events  
 WHERE processed = 1 
   AND eStatus != 'OK' 
   AND tStatus = 'NEW' 
   AND tSource IN ('source_1', 'source_2', 'source_3', 'source_4', 'source_5') 
   AND dateCreated BETWEEN '2015-10-01 10:36:00' AND  '2015-10-16 10:44:00' 
 ORDER 
    BY dateCreated DESC;

我遇到的问题是,如果查询完成,查询需要很长时间才能完成,并且在大多数情况下运行show processlist只会让查询停留在发送数据上,所以即使解释也不会再返回

最初我认为ORDER BY导致了一个问题,所以我这样做但它仍然无法正常工作

EXPLAIN 
SELECT * 
  FROM 
     ( SELECT ID AS id
            , tSource
            , eDescription
            , eName
            , dId
            , dIp
            , tRef
            , customerName
            , dName
            , ePri
            , dateCreated
            , tStatus
            , supp 
         FROM events e  
        WHERE processed = 1 
          AND eStatus != 'OK' 
          AND tStatus = 'NEW' 
          AND tSource IN ('source_1', 'source_2', 'source_3', 'source_4', 'source_5') 
          AND dateCreated BETWEEN '2015-10-01 10:36:00' AND  '2015-10-16 10:44:00'
     ) as e  
 ORDER  
    BY dateCreated DESC;

我还试图检查数据库是否有损坏,它恢复正常

更新:EXPLAIN现在返回

-+------------------------------------------+---------+-------------+--------+-----------------------------+
| id | select_type | table | type | possible_keys                                                                                                                        | key                                      | key_len | ref         | rows   | Extra                       |
+----+-------------+-------+------+--------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+---------+-------------+--------+-----------------------------+
|  1 | SIMPLE      | e     | ref  | status_customer_date_tstatus,event_suppress,eName_eStatus_dName_date_proc,proc_tStatus_customer_date_priority                        | proc_tStatus_customer_date_priority      | 21      | const,const | 996507 | Using where; Using filesort |
+----+-------------+-------+------+--------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+---------+-------------+--------+-----------------------------+

更新:分析显示大部分时间用于排序数据 - 我应该在代码中对此进行排序而不是MySQL事件,当我使用ORDER BY ID而不是DateCreated时(我认为MySQL能够相当快地完成此操作) - 为格式化道歉

mysql> SELECT * FROM INFORMATION_SCHEMA.PROFILING WHERE QUERY_ID=2;
    +----------+-----+--------------------------------+------------+-----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
| QUERY_ID | SEQ | STATE                          | DURATION   | CPU_USER  | CPU_SYSTEM | CONTEXT_VOLUNTARY | CONTEXT_INVOLUNTARY | BLOCK_OPS_IN | BLOCK_OPS_OUT | MESSAGES_SENT | MESSAGES_RECEIVED | PAGE_FAULTS_MAJOR | PAGE_FAULTS_MINOR | SWAPS | SOURCE_FUNCTION       | SOURCE_FILE   | SOURCE_LINE |
+----------+-----+--------------------------------+------------+-----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
|        2 |   2 | starting                       |   0.000029 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | NULL                  | NULL          |        NULL |
|        2 |   3 | Waiting for query cache lock   |   0.000008 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | try_lock              | sql_cache.cc  |         454 |
|        2 |   4 | checking query cache for query |   0.000101 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | send_result_to_client | sql_cache.cc  |        1561 |
|        2 |   5 | checking permissions           |   0.000010 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | check_access          | sql_parse.cc  |        4786 |
|        2 |   6 | Opening tables                 |   0.000020 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | open_tables           | sql_base.cc   |        4831 |
|        2 |   7 | System lock                    |   0.000010 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_lock_tables     | lock.cc       |         299 |
|        2 |   8 | Waiting for query cache lock   |   0.000061 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | try_lock              | sql_cache.cc  |         454 |
|        2 |   9 | init                           |   0.000040 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_select          | sql_select.cc |        2579 |
|        2 |  10 | optimizing                     |   0.000023 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | optimize              | sql_select.cc |         865 |
|        2 |  11 | statistics                     |   0.000259 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | optimize              | sql_select.cc |        1056 |
|        2 |  12 | preparing                      |   0.000028 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | optimize              | sql_select.cc |        1078 |
|        2 |  13 | executing                      |   0.000005 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | exec                  | sql_select.cc |        1836 |
|        2 |  14 | Sorting result                 | 999.999999 | 17.165073 |   5.588349 |             74361 |                1326 |      6687376 |        208944 |             0 |                 0 |                 1 |            111411 |     0 | exec                  | sql_select.cc |        2267 |
|        2 |  15 | Sending data                   |  57.755522 |  1.492093 |   0.364023 |              4331 |                 111 |       233040 |         10792 |             0 |                 0 |                 0 |              2842 |     0 | exec                  | sql_select.cc |        2380 |
|        2 |  16 | end                            |   0.000021 |  0.000000 |   0.000000 |                 1 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_select          | sql_select.cc |        2615 |
|        2 |  17 | query end                      |   0.000007 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_execute_command | sql_parse.cc  |        4465 |
|        2 |  18 | closing tables                 |   0.000014 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_execute_command | sql_parse.cc  |        4517 |
|        2 |  19 | freeing items                  |   0.000028 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_parse           | sql_parse.cc  |        5675 |
|        2 |  20 | logging slow query             |   0.000005 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | log_slow_statement    | sql_parse.cc  |        1479 |
|        2 |  21 | logging slow query             |   0.000063 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |            16 |             0 |                 0 |                 0 |                 0 |     0 | log_slow_statement    | sql_parse.cc  |        1488 |
|        2 |  22 | cleaning up                    |   0.000007 |  0.000000 |   0.000000 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | dispatch_command      | sql_parse.cc  |        1435 |
+----------+-----+--------------------------------+------------+-----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+

有什么想法吗?

感谢

3 个答案:

答案 0 :(得分:0)

首先,摆脱子查询;它只会增加开销:

 SELECT ID, tSource, eDescription, eName, dId, dIp, tRef, customerName,
        dName, ePri, dateCreated, tStatus, supp 
 FROM events e  
 WHERE processed = 1 AND
       eStatus <> 'OK' AND
       tStatus = 'NEW' AND
       tSource IN ('source_1', 'source_2', 'source_3', 'source_4', 'source_5') AND
       dateCreated BETWEEN '2015-10-01 10:36:00' AND  '2015-10-16 10:44:00' 
 ORDER BY dateCreated DESC;

此查询的最佳索引以processedtStatus开头。然后它应该有另一列,可能是dateCreated。因此,如果索引更具选择性,可能更快:events(processed, tStatus, dateCreated)。您可以在这些列之后引入eStatustSource ,因此索引会涵盖WHERE子句。

这不会使用order by的索引。您可以按id desc代替dateCreated desc订购吗?它是一个主键,因此是集群的(在MySQL中),因此您可以绕过最后的order by

答案 1 :(得分:0)

我同意删除子查询,但建议在以下上下文中创建覆盖索引

索引...... (已处理,tstatus,tsource,datecreated,estatus)

答案 2 :(得分:0)

INDEX(processed, tStatus, dateCreated) -- should be pretty good in any version
INDEX(processed, tStatus, tSource, dateCreated) -- might work best in a new 5.6