我有一个mysql的视图:
CREATE VIEW
loggingquarantine_quarantine ( id, mail_id, partition_tag, content, rs, subject, sender, TIME,
spam_level, size, sid, email ) AS
SELECT
concat(CAST(`mr`.`mail_id` AS CHAR(255) charset utf8),CAST(`mr`.`partition_tag` AS CHAR(255)
charset utf8)) AS `id`,
`mr`.`mail_id` AS `mail_id`,
`mr`.`partition_tag` AS `partition_tag`,
`mr`.`content` AS `content`,
`mr`.`rs` AS `rs`,
`m`.`subject` AS `subject`,
`m`.`from_addr` AS `sender`,
`m`.`time_num` AS `TIME`,
`m`.`spam_level` AS `spam_level`,
`m`.`size` AS `size`,
`m`.`sid` AS `sid`,
`maddr`.`email` AS `email`
FROM
(((`msgrcpt` `mr` JOIN `msgs` `m`
ON
(
`m`.`partition_tag` = `mr`.`partition_tag`
AND
`m`.`mail_id` = `mr`.`mail_id`
)
)
JOIN `maddr` maddr
ON
(
`mr`.`rid` = `maddr`.`id`
)
))
当我试图计算这个视图时,花费大约13分钟来获得250万的记录。那非常慢。所有字段都有索引。如果我指望每张桌子,它不会超过20秒。 这是mysql解释显示的内容:
mysql> explain SELECT COUNT(*) FROM `loggingquarantine_quarantine`;
+----+-------------+-------+--------+-----------------------------------------------------------------------+-------------------------+---------+-----------------------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-----------------------------------------------------------------------+-------------------------+---------+-----------------------------------------------------------+---------+-------------+
| 1 | SIMPLE | maddr | index | PRIMARY | maddr_partition_tag_idx | 5 | NULL | 1016497 | Using index |
| 1 | SIMPLE | mr | ref | PRIMARY,msgrcpt_idx_rid,msgrcpt_mail_id_idx,msgrcpt_partition_tag_idx | msgrcpt_idx_rid | 8 | mroute_logquar.maddr.id | 2 | Using index |
| 1 | SIMPLE | m | eq_ref | PRIMARY,msgs_mail_id_idx,msgs_partition_tag_idx | PRIMARY | 22 | mroute_logquar.mr.partition_tag,mroute_logquar.mr.mail_id | 1 | Using index |
+----+-------------+-------+--------+-----------------------------------------------------------------------+-------------------------+---------+-----------------------------------------------------------+---------+-------------+
如何优化查询/视图,以便不需要13分钟进行计数..当前查询有什么问题?
更新。 如果我在没有视图的情况下直接在select上执行选择计数,它仍会获得相同的14分钟查询。
mysql> select count(1) FROM (((`msgrcpt` `mr` JOIN `msgs` `m` ON ( `m`.`partition_tag` = `mr`.`partition_tag` AND `m`.`mail_id` = `mr`.`mail_id` ) ) JOIN `maddr` maddr ON ( `mr`.`rid` = `maddr`.`id` ) ));
+----------+
| count(1) |
+----------+
| 2582227 |
+----------+
1 row in set (14 min 28.96 sec)
当我在三个单独的查询中执行此操作时,这是计数的结果:
mysql> select count(1) from msgrcpt;
+----------+
| count(1) |
+----------+
| 2587307 |
+----------+
1 row in set (46.02 sec)
mysql> select count(1) from msgs;
+----------+
| count(1) |
+----------+
| 2421710 |
+----------+
1 row in set (7.77 sec)
mysql> select count(1) from maddr;
+----------+
| count(1) |
+----------+
| 994880 |
+----------+
1 row in set (0.23 sec)
更新2。
所有表格都是InnoDB。
mysql> SHOW status like 'key_%'; +------------------------+-------+
| Variable_name | Value |
+------------------------+-------+
| Key_blocks_not_flushed | 0 |
| Key_blocks_unused | 26792 |
| Key_blocks_used | 0 |
| Key_read_requests | 0 |
| Key_reads | 0 |
| Key_write_requests | 0 |
| Key_writes | 0 |
+------------------------+-------+
msgs和msgrcpt表具有复合主键(msgsm的ms_和partition_tag
,mail_id
,rseqnum
的mail_id,partation_tag)。
UPDATE
解释单表:
mysql> explain select count(1) from msgs;
+----+-------------+-------+-------+---------------+-------------------+---------+------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+-------------------+---------+------+---------+-------------+
| 1 | SIMPLE | msgs | index | NULL | msgs_idx_time_num | 4 | NULL | 2357360 | Using index |
+----+-------------+-------+-------+---------------+-------------------+---------+------+---------+-------------+
1 row in set (0.00 sec)
mysql> explain select count(1) from msgrcpt;
+----+-------------+---------+-------+---------------+----------------+---------+------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------+-------+---------------+----------------+---------+------+---------+-------------+
| 1 | SIMPLE | msgrcpt | index | NULL | msgrcpt_rs_idx | 3 | NULL | 2620758 | Using index |
+----+-------------+---------+-------+---------------+----------------+---------+------+---------+-------------+
1 row in set (0.00 sec)
mysql> explain select count(1) from maddr;
+----+-------------+-------+-------+---------------+-------------------------+---------+------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+-------------------------+---------+------+--------+-------------+
| 1 | SIMPLE | maddr | index | NULL | maddr_partition_tag_idx | 5 | NULL | 967058 | Using index |
+----+-------------+-------+-------+---------------+-------------------------+---------+------+--------+-------------+
1 row in set (0.00 sec)
更新。 为所有表创建表:
mysql> show create table msgrcpt;
| Table | Create Table
| msgrcpt | CREATE TABLE `msgrcpt` (
`partition_tag` int(11) NOT NULL DEFAULT '0',
`mail_id` varbinary(16) NOT NULL,
`rseqnum` int(11) NOT NULL DEFAULT '0',
`rid` bigint(20) unsigned NOT NULL,
`is_local` char(1) NOT NULL DEFAULT '',
`content` char(1) NOT NULL DEFAULT '',
`ds` char(1) NOT NULL,
`rs` char(1) NOT NULL,
`bl` char(1) DEFAULT '',
`wl` char(1) DEFAULT '',
`bspam_level` float DEFAULT NULL,
`smtp_resp` varchar(255) DEFAULT '',
PRIMARY KEY (`partition_tag`,`mail_id`,`rseqnum`),
KEY `msgrcpt_idx_rid` (`rid`),
KEY `msgrcpt_mail_id_idx` (`mail_id`),
KEY `msgrcpt_rs_idx` (`rs`),
KEY `msgrcpt_ds_idx` (`ds`),
KEY `msgrcpt_partition_tag_idx` (`partition_tag`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
| msgs | CREATE TABLE `msgs` (
`partition_tag` int(11) NOT NULL DEFAULT '0',
`mail_id` varbinary(16) NOT NULL,
`secret_id` varbinary(16) DEFAULT '',
`am_id` varchar(20) NOT NULL,
`time_num` int(10) unsigned NOT NULL,
`time_iso` char(16) NOT NULL,
`sid` bigint(20) unsigned NOT NULL,
`policy` varchar(255) DEFAULT '',
`client_addr` varchar(255) DEFAULT '',
`size` int(10) unsigned NOT NULL,
`originating` char(1) NOT NULL DEFAULT '',
`content` char(1) DEFAULT NULL,
`quar_type` char(1) DEFAULT NULL,
`quar_loc` varbinary(255) DEFAULT '',
`dsn_sent` char(1) DEFAULT NULL,
`spam_level` float DEFAULT NULL,
`message_id` varchar(255) DEFAULT '',
`from_addr` varchar(255) DEFAULT '',
`subject` varchar(255) DEFAULT '',
`host` varchar(255) NOT NULL,
PRIMARY KEY (`partition_tag`,`mail_id`),
KEY `msgs_idx_sid` (`sid`),
KEY `msgs_idx_mess_id` (`message_id`),
KEY `msgs_idx_time_num` (`time_num`),
KEY `msgs_mail_id_idx` (`mail_id`),
KEY `msgs_partition_tag_idx` (`partition_tag`),
KEY `msgs_content_idx` (`content`),
FULLTEXT KEY `ft_from_addr` (`from_addr`),
FULLTEXT KEY `ft_subject` (`subject`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
| maddr | CREATE TABLE `maddr` (
`partition_tag` int(11) DEFAULT '0',
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`email` varbinary(255) NOT NULL,
`domain` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `part_email` (`partition_tag`,`email`),
KEY `maddr_email_idx` (`email`),
KEY `maddr_partition_tag_idx` (`partition_tag`)
) ENGINE=InnoDB AUTO_INCREMENT=3373444 DEFAULT CHARSET=utf8 |
此查询包含个人资料:
mysql> SET PROFILING=1; SELECT
Query OK, 0 rows affected (0.00 sec)
-> count(1)
-> FROM
-> (((`msgrcpt` `mr` JOIN `msgs` `m`
-> ON
-> (
-> `m`.`partition_tag` = `mr`.`partition_tag`
-> AND
-> `m`.`mail_id` = `mr`.`mail_id`
-> )
-> )
-> JOIN `maddr` maddr
-> ON
-> (
-> `mr`.`rid` = `maddr`.`id`
-> )
-> )); SHOW PROFILE ALL;
+----------+
| count(1) |
+----------+
| 4279394 |
+----------+
1 row in set (23 min 56.61 sec)
+----------------------+------------+-----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+------------------+-------------+
| Status | Duration | CPU_user | CPU_system | Context_voluntary | Context_involuntary | Block_ops_in | Block_ops_out | Messages_sent | Messages_received | Page_faults_major | Page_faults_minor | Swaps | Source_function | Source_file | Source_line |
+----------------------+------------+-----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+------------------+-------------+
| starting | 0.000161 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NULL | NULL | NULL |
| checking permissions | 0.000030 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | check_access | sql_parse.cc | 5043 |
| checking permissions | 0.000019 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | check_access | sql_parse.cc | 5043 |
| checking permissions | 0.000020 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | check_access | sql_parse.cc | 5043 |
| Opening tables | 0.000039 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | open_tables | sql_base.cc | 5014 |
| System lock | 0.000026 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_lock_tables | lock.cc | 304 |
| init | 0.000040 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_select | sql_select.cc | 1041 |
| optimizing | 0.000030 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | optimize | sql_optimizer.cc | 138 |
| statistics | 0.000063 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | optimize | sql_optimizer.cc | 358 |
| preparing | 0.000032 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | optimize | sql_optimizer.cc | 470 |
| executing | 0.000021 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | exec | sql_executor.cc | 137 |
| Sending data | 999.999999 | 97.014251 | 10.376423 | 681167 | 25822 | 5157072 | 1951032 | 0 | 0 | 4 | 277 | 0 | execute | sql_executor.cc | 758 |
| end | 0.000106 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_select | sql_select.cc | 1071 |
| query end | 0.000017 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_execute_command | sql_parse.cc | 4761 |
| closing tables | 0.000021 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_execute_command | sql_parse.cc | 4809 |
| freeing items | 0.000030 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_parse | sql_parse.cc | 5997 |
| logging slow query | 0.000059 | 0.000000 | 0.000000 | 0 | 0 | 0 | 8 | 0 | 0 | 0 | 0 | 0 | log_slow_statement | sql_parse.cc | 1720 |
| cleaning up | 0.000019 | 0.000000 | 0.000000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | dispatch_command | sql_parse.cc | 1654 |
+----------------------+------------+-----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+------------------+-------------+
18 rows in set (0.02 sec)
表格索引:
mysql> show index from msgs;
+-------+------------+------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| msgs | 0 | PRIMARY | 1 | partition_tag | A | 16 | NULL | NULL | | BTREE | | |
| msgs | 0 | PRIMARY | 2 | mail_id | A | 4174440 | NULL | NULL | | BTREE | | |
| msgs | 1 | msgs_idx_sid | 1 | sid | A | 2087220 | NULL | NULL | | BTREE | | |
| msgs | 1 | msgs_idx_mess_id | 1 | message_id | A | 4174440 | NULL | NULL | YES | BTREE | | |
| msgs | 1 | msgs_idx_time_num | 1 | time_num | A | 1391480 | NULL | NULL | | BTREE | | |
| msgs | 1 | msgs_mail_id_idx | 1 | mail_id | A | 4174440 | NULL | NULL | | BTREE | | |
| msgs | 1 | msgs_partition_tag_idx | 1 | partition_tag | A | 16 | NULL | NULL | | BTREE | | |
| msgs | 1 | msgs_content_idx | 1 | content | A | 16 | NULL | NULL | YES | BTREE | | |
| msgs | 1 | ft_from_addr | 1 | from_addr | NULL | 4174440 | NULL | NULL | YES | FULLTEXT | | |
| msgs | 1 | ft_subject | 1 | subject | NULL | 4174440 | NULL | NULL | YES | FULLTEXT | | |
+-------+------------+------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
10 rows in set (0.97 sec)
MSGRCPT
mysql> show index from msgrcpt;
+---------+------------+---------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------+------------+---------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| msgrcpt | 0 | PRIMARY | 1 | partition_tag | A | 29 | NULL | NULL | | BTREE | | |
| msgrcpt | 0 | PRIMARY | 2 | mail_id | A | 5218535 | NULL | NULL | | BTREE | | |
| msgrcpt | 0 | PRIMARY | 3 | rseqnum | A | 5218535 | NULL | NULL | | BTREE | | |
| msgrcpt | 1 | msgrcpt_idx_rid | 1 | rid | A | 347902 | NULL | NULL | | BTREE | | |
| msgrcpt | 1 | msgrcpt_mail_id_idx | 1 | mail_id | A | 5218535 | NULL | NULL | | BTREE | | |
| msgrcpt | 1 | msgrcpt_rs_idx | 1 | rs | A | 29 | NULL | NULL | | BTREE | | |
| msgrcpt | 1 | msgrcpt_ds_idx | 1 | ds | A | 29 | NULL | NULL | | BTREE | | |
| msgrcpt | 1 | msgrcpt_partition_tag_idx | 1 | partition_tag | A | 29 | NULL | NULL | | BTREE | | |
+---------+------------+---------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
8 rows in set (0.70 sec)
MADDR:
mysql> show index from maddr;
+-------+------------+-------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+-------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| maddr | 0 | PRIMARY | 1 | id | A | 1653970 | NULL | NULL | | BTREE | | |
| maddr | 0 | part_email | 1 | partition_tag | A | 19 | NULL | NULL | YES | BTREE | | |
| maddr | 0 | part_email | 2 | email | A | 1653970 | NULL | NULL | | BTREE | | |
| maddr | 1 | maddr_email_idx | 1 | email | A | 1653970 | NULL | NULL | | BTREE | | |
| maddr | 1 | maddr_partition_tag_idx | 1 | partition_tag | A | 19 | NULL | NULL | YES | BTREE | | |
+-------+------------+-------------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
5 rows in set (0.41 sec)
Inno db buffer size
mysql> SHOW VARIABLES LIKE 'innodb_buffer_pool_size';
+-------------------------+------------+
| Variable_name | Value |
+-------------------------+------------+
| innodb_buffer_pool_size | 2147483648 |
+-------------------------+------------+
1 row in set (0.02 sec)
答案 0 :(得分:1)
您是否尝试过使用某些STRAIGHT_JOIN而不是简单的JOIN来改变JOIN中使用的表的顺序?有时,查询优化器不会为特定查询选择理想的顺序。换句话说,你对此有何看法?
SELECT
concat(CAST(`mr`.`mail_id` AS CHAR(255) charset utf8),CAST(`mr`.`partition_tag` AS CHAR(255) charset utf8)) AS `id`,
`mr`.`mail_id` AS `mail_id`,
`mr`.`partition_tag` AS `partition_tag`,
`mr`.`content` AS `content`,
`mr`.`rs` AS `rs`,
`m`.`subject` AS `subject`,
`m`.`from_addr` AS `sender`,
`m`.`time_num` AS `TIME`,
`m`.`spam_level` AS `spam_level`,
`m`.`size` AS `size`,
`m`.`sid` AS `sid`,
`maddr`.`email` AS `email`
FROM
`msgrcpt` `mr`
STRAIGHT_JOIN `msgs` `m`
ON
`m`.`partition_tag` = `mr`.`partition_tag` AND
`m`.`mail_id` = `mr`.`mail_id`
STRAIGHT_JOIN `maddr` maddr
ON
`mr`.`rid` = `maddr`.`id`
或者,因为你在msgrcpt表上使用了一些重型转换和转换,如果强制它在查询中最后一次加入会发生什么呢?
SELECT
concat(CAST(`mr`.`mail_id` AS CHAR(255) charset utf8),CAST(`mr`.`partition_tag` AS CHAR(255) charset utf8)) AS `id`,
`mr`.`mail_id` AS `mail_id`,
`mr`.`partition_tag` AS `partition_tag`,
`mr`.`content` AS `content`,
`mr`.`rs` AS `rs`,
`m`.`subject` AS `subject`,
`m`.`from_addr` AS `sender`,
`m`.`time_num` AS `TIME`,
`m`.`spam_level` AS `spam_level`,
`m`.`size` AS `size`,
`m`.`sid` AS `sid`,
`maddr`.`email` AS `email`
FROM
`maddr` `maddr`
STRAIGHT_JOIN `msgrcpt` `mr`
ON
`maddr`.`id` = `mr.`rid`
STRAIGHT_JOIN `msgs` `m`
ON
`m`.`partition_tag` = `mr`.`partition_tag` AND
`m`.`mail_id` = `mr`.`mail_id`
如果需要,您可以更多地使用订单,可能会有一个“魔术订单”导致您的查询压缩。例如,另一种变体可能是:
FROM
`msgs` `m`
STRAIGHT_JOIN `msgrcpt` `mr`
ON
`m`.`partition_tag` = `mr`.`partition_tag` AND
`m`.`mail_id` = `mr`.`mail_id`
STRAIGHT_JOIN `maddr`
ON
`maddr`.`id` = `mr`.`rid`
......等等。
此外,在你的SELECT列中,所有这些强制转换都会导致你的性能下降,因为无论你选择多少行都必须这样做,这可能会很多。您是否考虑过将一列添加到已经计算过的msgrcpt表中,以便它不必在查询中执行此操作?您可以向数据库添加触发器,以便有效地自动更新它,如果您不想在代码中考虑额外列,则无需使用计算值。通常情况下,使用这种优化有点过分,但在对数百万行进行SELECT查询时,它可能就是这样做的。
编辑:以下是我建议您更改msgrcpt表格的内容。它会使插入行的速度变慢很小,因为它运行一个触发器来计算新的列值,但我认为它会在加速你试图运行的查询时获得相当大的回报。标准免责声明适用 - 如果没有在测试副本上进行彻底测试,请不要在您的生产数据库上运行此文件!
-- The UPDATE command will probably take some time to run since it's updating
-- millions of rows. Be patient!
ALTER TABLE `msgrcpt` ADD COLUMN `friendly_id` TEXT NULL AFTER `rs`;
UPDATE `msgrcpt` SET
`friendly_id` = CONCAT(CAST(`mail_id` AS CHAR(255) CHARSET utf8),
CAST(`partition_tag` AS CHAR(255) CHARSET utf8));
DELIMITER $$
CREATE TRIGGER `trig_calc_id` BEFORE INSERT ON `msgrcpt`
FOR EACH ROW BEGIN
SET NEW.`friendly_id` =
CONCAT(CAST(NEW.`mail_id` AS CHAR(255) CHARSET utf8),
CAST(NEW.`partition_tag` AS CHAR(255) CHARSET utf8));
END $$
CREATE TRIGGER `trig_update_id` BEFORE UPDATE ON `msgrcpt`
FOR EACH ROW BEGIN
SET NEW.`friendly_id` =
CONCAT(CAST(NEW.`mail_id` AS CHAR(255) CHARSET utf8),
CAST(NEW.`partition_tag` AS CHAR(255) CHARSET utf8));
END $$
DELIMITER ;
现在不要选择那些乱糟糟的CONCATed CASTed东西,只需选择mr.friendly_id(或者你选择命名列的任何东西)。你的表现要好得多。
希望这会有所帮助,让我知道它是怎么回事!
答案 1 :(得分:0)
several methods数据库如何执行表连接。 MySQL正在使用 Nested-loops加入,如section on EXPLAIN
output中所述。嵌套循环的工作方式是理解为什么这个查询花费这么多时间的关键。
根据您的EXPLAIN
输出,MySQL首先扫描maddr
表,因此您有1 016 497
个执行循环。在每个执行数据库中,在两个小表上执行索引扫描。即使费用:
非常小(因为数据很可能将被许多执行缓存),无论如何都需要几毫秒。然后乘以2
,msgrcpt
和msgs
都在内部扫描。然后乘以外循环的执行次数,这给你很长的时间。
您的视图定义没有对返回的数据进行过滤,因此必须扫描和连接所有表。鉴于嵌套循环如何工作,很明显,当外部查询包含最少量的循环时,变量是性能最佳的循环。考虑到所有表的大小,索引扫描(根据EXPLAIN
输出在索引上扫描所有3个表)将进行检查:
msgrcpt
:log 2 (2587307)+ 1 = 22页,msgs
:log 2 (2421710)+ 1 = 22页maddr
:log 2 (994880)+ 1 =每次扫描21页。这表明表扫描时间具有可比性,即使使用索引扫描也非常重要。尽管如此,maddr
更小,这就是MySQL首先选择扫描它的原因。
正如我所看到的,数据库选择的计划是给定情况下的最佳计划。并且MySQL仅支持嵌套循环连接的事实没有提供优化可能性,因为外部循环已经是最小的。此处可能会使用Hash join来获得更好的效果。
那么,如何让事情在这种情况下表现呢?
我建议仅使用视图来查询带过滤器的数据,这将使MySQL有更多机会优化查询并快速返回数据。
要获得行数,我会去估算(如果这是可接受的),计算每个表的count(*)
并取最大值。
另一种方法是重新设计模式,使其中一个表成为主模式,以便视图中的行数始终与主表中的行数相匹配。对不起,我不知道如何用给定的设计做到这一点。
如果没有一个选项可以接受,遗憾的是,我认为没有其他方法可以优化此查询。
答案 2 :(得分:0)
我同意Skippus国王关于两个要素...... STRAIGHT_JOIN帮助推翻优化器执行计划......以及为您保留预聚合的触发器。但是,如果您想要应用WHERE子句进行限制,则会被枪杀。
此外,如果您关心的只是一个COUNT,那么vyegorov评论抛出所有列将是最好的。通过让所有其他列强制从所有实际数据页中提取数据。
我会将查询呈现为
SELECT STRAIGHT_JOIN COUNT(*)
FROM
msgrcpt mr
JOIN msgs m
ON mr.mail_id = m.mail_id
AND mr.partition_tag = m.partition_tag
JOIN maddr
ON mr.rid = maddr.id
我知道你提到索引,但我会 MsgRcpt表索引通过(mail_id,partition_tag) Msgs表索引通过相同的(mail_id,partition_tag) MAddr索引(id)
现在,为什么索引上的另一种方式与另一种方式相比?我会尝试在第一个索引字段中基于最小聚合。不确切知道您的卷基础是什么(邮件ID,分区标记),而是获取客户订单系统的样本。你可以让一个人在一天内订购一些东西,但对于一个给定的人,在某一天订购一两个订单。首先找到该人,将该个人日期与单日相比更容易找到一个人。这可能会对尝试首先通过LARGE列表的查询性能产生影响,然后是小型与小型/小型。
只需要一个COUNT(*),就可以得到一个计数。原始表中的其他列不是必需的,并且每个页面都被加载,因为它只需要索引的组件来限定结果中的记录。
如果你想要所有的数据,那么我会做同样的索引考虑因素,但查询为......
CREATE VIEW
loggingquarantine_quarantine ( id, mail_id, partition_tag, content, rs, subject, sender, TIME,
spam_level, size, sid, email ) AS
SELECT STRAIGHT_JOIN
concat(CAST(mr.mail_id AS CHAR(255) charset utf8),
CAST(mr.partition_tag AS CHAR(255) charset utf8)) AS `id`,
mr.mail_id,
mr.partition_tag,
mr.content,
mr.rs,
m.subject,
m.from_addr AS sender,
m.time_num AS `TIME`,
m.spam_level,
m.size,
m.sid,
maddr.email
FROM
msgrcpt mr
JOIN msgs m
ON mr.mail_id = m.mail_id
AND mr.partition_tag = m.partition_tag
JOIN maddr
ON mr.rid = maddr.id
答案 3 :(得分:0)
视图查询itselt以及代码中视图的使用可能会影响性能。有关详细信息,请阅读文章 - MySQL VIEW as performance troublemaker。
解决方法:
答案 4 :(得分:0)
尝试使用COUNT(*)insted COUNT(1),因为MySQL优化器非常错误 http://www.mysqlperformanceblog.com/2007/04/10/count-vs-countcol/
答案 5 :(得分:0)
对不起:我看到你的maddr表有一个partition_tag id,但是在组成VIEW的最后一个JOIN中没有使用该分区标记。这是应该的吗?
无论如何,在我看来,您的查询可能会受益于包含id和email的maddr索引。这应该允许直接从索引中提取具有较少I / O的电子邮件(这可能有利于评估各种索引的性能 - 在我看来,例如,单独的电子邮件字段中的索引很少会被使用,但它取决于你知道的工作流程,我不知道,所以我很可能会弄错。
答案 6 :(得分:0)
在count()
中写一个主键列,使其更快。像count(id)
答案 7 :(得分:0)
好吧:),所以一方面你说 - 我们有250万个条目,另一方面(根据你)计数返回4279394(略高:) :)。可能的原因(对不起,差不多凌晨3点:)) - 你没有指定msgrcpt的完整主键(rseqnum缺失),所以在优化器选择maddr或msgs作为主表的情况下 - 你可能会得到相同的结果msgrcpt的条目。同样基于msgrcpt和msgs的计数时间的差异,我强烈建议对msgrcpt进行碎片整理。我相信在对直接连接进行碎片整理和使用(已经推荐)之后,你的问题就会消失(如果没有 - 也许你可以找到一种方法来动态计算 - 无论你需要为rseqnum / spam_level计算什么,或者,如果spam_level是相同的完整的信息 - 也许你可以通过提供rseqnum的硬编码值来限制选择。
答案 8 :(得分:0)
加入需要BIG mysql加载。 在我的经验中, 创建临时表并从中选择数据, 提高数据库查询速度。
这是我的代码,代码2显示了100倍的快速速度。 您可以像代码2样式一样更改查询代码。 对于大型复杂的数据库查询,请使用临时表提升数据库查询速度。
代码1
$sql = " select distinct wr_parent from $write_table where $sql_search ";
$result = sql_query($sql);
$total_count = mysql_num_rows($result);
代码2
$sql = " select wr_parent from $write_table where $sql_search ";
$sql_tmp = " create TEMPORARY table list_tmp_count as $sql ";
$sql_ord = " select distinct wr_parent from list_tmp_count ";
@mysql_query($sql_tmp) or die("<p>$sql_tmp<p>" . mysql_errno() . " : " . mysql_error() . "<p>error file : $_SERVER[PHP_SELF]");
$result = @mysql_query($sql_ord) or die("<p>$sql_ord<p>" . mysql_errno() . " : " . mysql_error() . "<p>error file : $_SERVER[PHP_SELF]");
$total_count = mysql_num_rows($result);
如果临时表在tempfs(内存区域)中的位置速度也加倍。