相同的查询,相同的系统,不同的执行时间

时间:2012-05-30 14:14:25

标签: mysql sql doctrine

以下查询都会立即在我们的开发服务器上执行,因为它们最多可能需要2分20秒。

查询执行时间似乎受到LIKE字符串所在的家庭模糊的影响。如果他们与匹配较少的国家匹配,则需要的时间较少,如果您对德国使用“ge”之类的东西,则执行时间会更长。但这并不总是那样,有时它很不稳定。

发送数据似乎是罪魁祸首但是为什么以及这意味着什么。生产中的内存看起来很低(可用内存)?

生产
英特尔四核至强E3-1220 3.1GHz
4GB DDR3
RAID1中的2x 1TB SATA
网速100Mb
Ubuntu

开发
英特尔酷睿i3-2100,2C / 4T,3.10GHz
500 GB SATA - 无RAID
4GB DDR3

这个查询不是有问题的查询,而是相关的,因为它很容易发布。

SELECT 
    f.form_question_has_answer_id 
FROM 
    form_question_has_answer f 
INNER JOIN 
    project_company_has_user p ON f.form_question_has_answer_user_id = p.project_company_has_user_user_id 
INNER JOIN 
    company c ON p.project_company_has_user_company_id = c.company_id 
INNER JOIN 
    project p2 ON p.project_company_has_user_project_id = p2.project_id 
INNER JOIN 
    user u ON p.project_company_has_user_user_id = u.user_id 
INNER JOIN 
    form f2 ON p.project_company_has_user_project_id = f2.form_project_id 
WHERE 
    (f2.form_template_name = 'custom' AND p.project_company_has_user_garbage_collection = 0 AND p.project_company_has_user_project_id = '29') AND (LCASE(c.company_country) LIKE '%ge%' OR LCASE(c.company_country) LIKE '%abcde%') AND f.form_question_has_answer_form_id = '174'

上述查询的解释计划是,在dev和production上运行产生相同的计划。

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+----------------------------------------------------+------+-------------+
| id | select_type | table | type   | possible_keys                                                                                                                                | key                              | key_len | ref                                                | rows | Extra       |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+----------------------------------------------------+------+-------------+
|  1 | SIMPLE      | p2    | const  | PRIMARY                                                                                                                                      | PRIMARY                          | 4       | const                                              |    1 | Using index |
|  1 | SIMPLE      | f     | ref    | form_question_has_answer_form_id,form_question_has_answer_user_id                                                                            | form_question_has_answer_form_id | 4       | const                                              |  796 | Using where |
|  1 | SIMPLE      | u     | eq_ref | PRIMARY                                                                                                                                      | PRIMARY                          | 4       | new_klarents.f.form_question_has_answer_user_id    |    1 | Using index |
|  1 | SIMPLE      | p     | ref    | project_company_has_user_unique_key,project_company_has_user_user_id,project_company_has_user_company_id,project_company_has_user_project_id | project_company_has_user_user_id | 4       | new_klarents.f.form_question_has_answer_user_id    |    1 | Using where |
|  1 | SIMPLE      | f2    | ref    | form_project_id                                                                                                                              | form_project_id                  | 4       | const                                              |   15 | Using where |
|  1 | SIMPLE      | c     | eq_ref | PRIMARY                                                                                                                                      | PRIMARY                          | 4       | new_klarents.p.project_company_has_user_company_id |    1 | Using where |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+----------------------------------------------------+------+-------------+

此查询需要2分钟~20秒才能执行。

在服务器上运行的查询是这样的:

SELECT 
    COUNT(*) AS num_results 
FROM (SELECT 
        f.form_question_has_answer_id 
    FROM 
        form_question_has_answer f 
    INNER JOIN 
        project_company_has_user p ON f.form_question_has_answer_user_id = p.project_company_has_user_user_id 
    INNER JOIN 
        company c ON p.project_company_has_user_company_id = c.company_id 
    INNER JOIN 
        project p2 ON p.project_company_has_user_project_id = p2.project_id 
    INNER JOIN 
        user u ON p.project_company_has_user_user_id = u.user_id 
    INNER JOIN 
        form f2 ON p.project_company_has_user_project_id = f2.form_project_id 
    WHERE 
        (f2.form_template_name = 'custom' AND p.project_company_has_user_garbage_collection = 0 AND p.project_company_has_user_project_id = '29') AND (LCASE(c.company_country) LIKE '%ge%' OR LCASE(c.company_country) LIKE '%abcde%') AND f.form_question_has_answer_form_id = '174' 
    GROUP BY 
        f.form_question_has_answer_id;) dctrn_count_query;

使用解释计划(在开发和生产方面同样相同):

+----+-------------+-------+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+----------------------------------------------------+------+------------------------------+
    | id | select_type | table | type   | possible_keys                                                                                                                                                                            | key                              | key_len | ref                                                | rows | Extra                        |
    +----+-------------+-------+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+----------------------------------------------------+------+------------------------------+
    |  1 | PRIMARY     | NULL  | NULL   | NULL                                                                                                                                                                                     | NULL                             | NULL    | NULL                                               | NULL | Select tables optimized away |
    |  2 | DERIVED     | p2    | const  | PRIMARY                                                                                                                                                                                  | PRIMARY                          | 4       |                                                    |    1 | Using index                  |
    |  2 | DERIVED     | f     | ref    | form_question_has_answer_form_id,form_question_has_answer_user_id                                                                                                                        | form_question_has_answer_form_id | 4       |                                                    |  797 | Using where                  |
    |  2 | DERIVED     | p     | ref    | project_company_has_user_unique_key,project_company_has_user_user_id,project_company_has_user_company_id,project_company_has_user_project_id,project_company_has_user_garbage_collection | project_company_has_user_user_id | 4       | new_klarents.f.form_question_has_answer_user_id    |    1 | Using where                  |
    |  2 | DERIVED     | f2    | ref    | form_project_id                                                                                                                                                                          | form_project_id                  | 4       |                                                    |   15 | Using where                  |
    |  2 | DERIVED     | c     | eq_ref | PRIMARY                                                                                                                                                                                  | PRIMARY                          | 4       | new_klarents.p.project_company_has_user_company_id |    1 | Using where                  |
    |  2 | DERIVED     | u     | eq_ref | PRIMARY                                                                                                                                                                                  | PRIMARY                          | 4       | new_klarents.p.project_company_has_user_user_id    |    1 | Using where; Using index     |
    +----+-------------+-------+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+----------------------------------------------------+------+------------------------------+

在生产服务器上,我的信息如下。

执行时:

+-------------+
| num_results |
+-------------+
|           3 |
+-------------+
1 row in set (2 min 14.28 sec)

显示个人资料:

+--------------------------------+------------+
| Status                         | Duration   |
+--------------------------------+------------+
| starting                       |   0.000016 |
| checking query cache for query |   0.000057 |
| Opening tables                 |   0.004388 |
| System lock                    |   0.000003 |
| Table lock                     |   0.000036 |
| init                           |   0.000030 |
| optimizing                     |   0.000016 |
| statistics                     |   0.000111 |
| preparing                      |   0.000022 |
| executing                      |   0.000004 |
| Sorting result                 |   0.000002 |
| Sending data                   | 136.213836 |
| end                            |   0.000007 |
| query end                      |   0.000002 |
| freeing items                  |   0.004273 |
| storing result in query cache  |   0.000010 |
| logging slow query             |   0.000001 |
| logging slow query             |   0.000002 |
| cleaning up                    |   0.000002 |
+--------------------------------+------------+

在开发方面,结果如下。

+-------------+
| num_results |
+-------------+
|           3 |
+-------------+
1 row in set (0.08 sec)

此查询的个人资料:

+--------------------------------+----------+
| Status                         | Duration |
+--------------------------------+----------+
| starting                       | 0.000022 |
| checking query cache for query | 0.000148 |
| Opening tables                 | 0.000025 |
| System lock                    | 0.000008 |
| Table lock                     | 0.000101 |
| optimizing                     | 0.000035 |
| statistics                     | 0.001019 |
| preparing                      | 0.000047 |
| executing                      | 0.000008 |
| Sorting result                 | 0.000005 |
| Sending data                   | 0.086565 |
| init                           | 0.000015 |
| optimizing                     | 0.000006 |
| executing                      | 0.000020 |
| end                            | 0.000004 |
| query end                      | 0.000004 |
| freeing items                  | 0.000028 |
| storing result in query cache  | 0.000005 |
| removing tmp table             | 0.000008 |
| closing tables                 | 0.000008 |
| logging slow query             | 0.000002 |
| cleaning up                    | 0.000005 |
+--------------------------------+----------+

如果我删除用户和/或项目内部连接,则查询将减少到30秒。

我的最后一点信息:

Mysqlserver和Apache在同一个盒子里,只有一个盒子可供生产。

顶部的产量:之前和之后后。

top - 15:43:25 up 78 days, 12:11,  4 users,  load average: 1.42, 0.99, 0.78
Tasks: 162 total,   2 running, 160 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us, 50.4%sy,  0.0%ni, 49.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4037868k total,  3772580k used,   265288k free,   243704k buffers
Swap:  3905528k total,   265384k used,  3640144k free,  1207944k cached

top - 15:44:31 up 78 days, 12:13,  4 users,  load average: 1.94, 1.23, 0.87
Tasks: 160 total,   2 running, 157 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.2%us, 50.6%sy,  0.0%ni, 49.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4037868k total,  3834300k used,   203568k free,   243736k buffers
Swap:  3905528k total,   265384k used,  3640144k free,  1207804k cached

但这不是生产正常状态的良好表现,所以这里是从今天开始执行查询之外的它。

top - 11:04:58 up 79 days,  7:33,  4 users,  load average: 0.39, 0.58, 0.76
Tasks: 156 total,   1 running, 155 sleeping,   0 stopped,   0 zombie
Cpu(s):  3.3%us,  2.8%sy,  0.0%ni, 93.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4037868k total,  3676136k used,   361732k free,   271480k buffers
Swap:  3905528k total,   268736k used,  3636792k free,  1063432k cached

开发:这个开发期间或之后不会发生变化。

top - 15:47:07 up 110 days, 22:11,  7 users,  load average: 0.17, 0.07, 0.06
Tasks: 210 total,   2 running, 208 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4111972k total,  1821100k used,  2290872k free,   238860k buffers
Swap:  4183036k total,    66472k used,  4116564k free,   921072k cached

7 个答案:

答案 0 :(得分:1)

如果缓慢是间歇性的,则可能是服务器负载或其他资源争用(在您的情况下,很可能是内存)。您的系统需要有足够的RAM来同时将所有索引存储在内存中,否则,如果所需的索引尚未加载到RAM中,则必须换掉内存。

您的TOP结果显示RAM可用量很低。

使用innodb_buffer_pool_size设置为InnoDB配置缓冲池的大小,为MyISAM配置key_buffer_size。确保将其设置得足够高,以便同时保存RAM中的所有索引,并确保系统有足够的RAM来完成此操作。

答案 1 :(得分:0)

发送数据| 136.213836

显然:D

这必须是某种基础设施问题...网络或其他什么,尝试从你的sql / plus终端ping你的服务器,你会得到你的答案

答案 2 :(得分:0)

解释计划通常是每当您查询速度较慢时启动的最佳位置。要获得一个,请运行

DESCRIBE SELECT 
    COUNT(*) AS num_results 
FROM (SELECT 
        f.form_question_has_answer_id 
    FROM 
        form_question_has_answer f 
    INNER JOIN 
        project_company_has_user p ON f.form_question_has_answer_user_id = p.project_company_has_user_user_id 
    INNER JOIN 
        company c ON p.project_company_has_user_company_id = c.company_id 
    INNER JOIN 
        project p2 ON p.project_company_has_user_project_id = p2.project_id 
    INNER JOIN 
        user u ON p.project_company_has_user_user_id = u.user_id 
    INNER JOIN 
        form f2 ON p.project_company_has_user_project_id = f2.form_project_id 
    WHERE 
        (f2.form_template_name = 'custom' AND p.project_company_has_user_garbage_collection = 0 AND p.project_company_has_user_project_id = '29') AND (LCASE(c.company_country) LIKE '%finland%' OR LCASE(c.company_country) LIKE '%finnland%') AND f.form_question_has_answer_form_id = '174' 
    GROUP BY 
        f.form_question_has_answer_id;) dctrn_count_query;

这将显示一个表,列出执行查询所需的步骤。如果您在'rows'列中看到一个较大的值而在'key'列中看到NULL,则表示您的查询必须扫描大量行以确定要返回的行。

在这种情况下,在destination_id上添加索引可以大大加快查询速度,但需要花费一些成本来插入和删除速度(因为索引也需要更新)。

答案 3 :(得分:0)

如果link正确,“发送数据”背后的数字不是发送数据所需的时间,而是“排序结果”。

这反过来可能暗示生产服务器上的内存或CPU问题。您可能希望查看机器上的相关统计信息。

答案 4 :(得分:0)

在开发和生产中运行查询并比较查询计划。如果它们不同,请尝试更新查询中涉及的表的统计信息。

答案 5 :(得分:0)

我唯一想到的是两台MySQL服务器之间的配置差异。两台服务器之间的内存设置有什么显着差异吗?它是虚拟服务器吗?

此外,生产服务器上的负载是否非常高?

答案 6 :(得分:0)

我对mysql中的优化器并不熟悉。

在查询结束时出现此问题。一种可能性是生产系统上出现的外观。例如,可能存在某种对元数据的锁定,这会阻止创建新表。

此外,查询运行的环境会影响优化程序。至少,多个线程/处理器将对查询计划产生影响。 Mysql可以根据可用资源生成不同的查询优化。我知道SQL Server将根据可用内存生成不同的查询计划 - 当有足够的内存时使用哈希表,但是当可用内存较少时使用嵌套的环连接。两个系统上的内存分配是否完全相同?