MySQL负载繁重的性能问题

时间:2016-06-22 10:26:42

标签: mysql sql performance sql-tuning

基本上,我们有一个很大的生产问题,我们的sql查询在结束前需要超过300秒。

我们使用Mysql作为SQL服务器,我们的表使用MyISAM。

有问题的查询是两个表之间的基本查询,以及排序和分组。问题是每个表都有超过3000万行。

以下是此查询的错误示例:

SELECT    
    i.id_stat as idStat,    
    SUM(srv0.duree_st) AS dureeAppelTotale,    
    _date as dateI,    
    DATE_FORMAT(srv0._date,'%d/%m') AS datekey,    
    DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,    
    heure,    
    SUM(srv0.nombre_st) AS nbAppel 
FROM    ids_stat i   , SRV201606 srv0  
WHERE    i.id_rubrique = srv0.id_rubrique    
    AND i.id_appli = 24071    
    AND srv0._date >= '2016-06-01 00:00:00'    
    AND srv0._date <= '2016-07-01 00:00:00'  
GROUP BY    id_stat, datekeyGroup, heure  
ORDER BY _date

ids_stat表上的索引是:

show index from ids_stat;
+----------+------------+---------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| Table    | Non_unique | Key_name            | Seq_in_index | Column_name   | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------+------------+---------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| ids_stat |          0 | PRIMARY             |            1 | id_rubrique   | A         |     7435537 |     NULL | NULL   |      | BTREE      |         |
| ids_stat |          1 | id_appli_index      |            1 | id_appli      | A         |       13593 |     NULL | NULL   |      | BTREE      |         |
| ids_stat |          1 | date_creation_index |            1 | date_creation | A         |     2478512 |     NULL | NULL   |      | BTREE      |         |
SRV0表上的

,索引是:

show index from SRV201606;
+-----------+------------+----------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+
| Table     | Non_unique | Key_name       | Seq_in_index | Column_name    | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------+------------+----------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+
| SRV201606 |          0 | PRIMARY        |            1 | _date          | A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| SRV201606 |          0 | PRIMARY        |            2 | heure          | A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| SRV201606 |          0 | PRIMARY        |            3 | minute         | A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| SRV201606 |          0 | PRIMARY        |            4 | id_plaque      | A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| SRV201606 |          0 | PRIMARY        |            5 | id_rubrique    | A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| SRV201606 |          0 | PRIMARY        |            6 | id_combinaison | A         |    27472765 |     NULL | NULL   |      | BTREE      |         |
| SRV201606 |          1 | id_rubrique    |            1 | id_rubrique    | A         |        NULL |     NULL | NULL   |      | BTREE      |         |
| SRV201606 |          1 | id_combinaison |            1 | id_combinaison | A         |        NULL |     NULL | NULL   |      | BTREE      |         |

此外,这是对查询的解释:

+----+-------------+-------+------+------------------------+----------------+---------+-----------------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys          | key            | key_len | ref                         | rows | Extra                           |
+----+-------------+-------+------+------------------------+----------------+---------+-----------------------------+------+---------------------------------+
|  1 | SIMPLE      | i     | ref  | PRIMARY,id_appli_index | id_appli_index | 2       | const                       |  153 | Using temporary; Using filesort |
|  1 | SIMPLE      | srv0  | ref  | PRIMARY,id_rubrique    | id_rubrique    | 8       | StatVoxProdV2.i.id_rubrique |   16 | Using where                     |

我们还读到这些旧的连接在哪里是无害的,因为它们被sql引擎转录为内部连接。

你能帮我们一些方法调整我们的mysql服务器,以便它可以提升吗?有没有可以帮助我们的配置?

非常感谢

[编辑] 正如评论中所说,这是他的版本:

SELECT VERSION();

+------------+
| VERSION()  |
+------------+
| 5.0.45-log |

服务器变量:

show Variables;
+---------------------------------+----------------------------------+
| Variable_name                   | Value                            |
+---------------------------------+----------------------------------+
| auto_increment_increment        | 1                                |
| auto_increment_offset           | 1                                |
| automatic_sp_privileges         | ON                               |
| back_log                        | 50                               |
| basedir                         | /usr/local/                      |
| binlog_cache_size               | 32768                            |
| bulk_insert_buffer_size         | 8388608                          |
| character_set_client            | utf8                             |
| character_set_connection        | utf8                             |
| character_set_database          | latin1                           |
| character_set_filesystem        | binary                           |
| character_set_results           | utf8                             |
| character_set_server            | latin1                           |
| character_set_system            | utf8                             |
| character_sets_dir              | /usr/local/share/mysql/charsets/ |
| collation_connection            | utf8_general_ci                  |
| collation_database              | latin1_swedish_ci                |
| collation_server                | latin1_swedish_ci                |
| completion_type                 | 0                                |
| concurrent_insert               | 1                                |
| connect_timeout                 | 5                                |
| datadir                         | THEDATADIR            |
| date_format                     | %Y-%m-%d                         |
| datetime_format                 | %Y-%m-%d %H:%i:%s                |
| default_week_format             | 0                                |
| delay_key_write                 | ON                               |
| delayed_insert_limit            | 100                              |
| delayed_insert_timeout          | 300                              |
| delayed_queue_size              | 1000                             |
| div_precision_increment         | 4                                |
| engine_condition_pushdown       | OFF                              |
| expire_logs_days                | 32                               |
| flush                           | OFF                              |
| flush_time                      | 0                                |
| ft_boolean_syntax               | + -><()~*:""&|                   |
| ft_max_word_len                 | 84                               |
| ft_min_word_len                 | 4                                |
| ft_query_expansion_limit        | 20                               |
| ft_stopword_file                | (built-in)                       |
| group_concat_max_len            | 1024                             |
| have_archive                    | NO                               |
| have_bdb                        | NO                               |
| have_blackhole_engine           | NO                               |
| have_compress                   | YES                              |
| have_crypt                      | YES                              |
| have_csv                        | NO                               |
| have_dynamic_loading            | YES                              |
| have_example_engine             | NO                               |
| have_federated_engine           | NO                               |
| have_geometry                   | YES                              |
| have_innodb                     | DISABLED                         |
| have_isam                       | NO                               |
| have_merge_engine               | YES                              |
| have_ndbcluster                 | NO                               |
| have_openssl                    | NO                               |
| have_ssl                        | NO                               |
| have_query_cache                | YES                              |
| have_raid                       | NO                               |
| have_rtree_keys                 | YES                              |
| have_symlink                    | YES                              |
| hostname                        | THEHOSTNAME                 |
| init_connect                    |                                  |
| init_file                       |                                  |
| init_slave                      |                                  |
| innodb_additional_mem_pool_size | 1048576                          |
| innodb_autoextend_increment     | 8                                |
| innodb_buffer_pool_awe_mem_mb   | 0                                |
| innodb_buffer_pool_size         | 1048576                          |
| innodb_checksums                | ON                               |
| innodb_commit_concurrency       | 0                                |
| innodb_concurrency_tickets      | 500                              |
| innodb_data_file_path           |                                  |
| innodb_data_home_dir            |                                  |
| innodb_doublewrite              | ON                               |
| innodb_fast_shutdown            | 1                                |
| innodb_file_io_threads          | 4                                |
| innodb_file_per_table           | OFF                              |
| innodb_flush_log_at_trx_commit  | 1                                |
| innodb_flush_method             |                                  |
| innodb_force_recovery           | 0                                |
| innodb_lock_wait_timeout        | 50                               |
| innodb_locks_unsafe_for_binlog  | OFF                              |
| innodb_log_arch_dir             |                                  |
| innodb_log_archive              | OFF                              |
| innodb_log_buffer_size          | 1048576                          |
| innodb_log_file_size            | 5242880                          |
| innodb_log_files_in_group       | 2                                |
| innodb_log_group_home_dir       |                                  |
| innodb_max_dirty_pages_pct      | 90                               |
| innodb_max_purge_lag            | 0                                |
| innodb_mirrored_log_groups      | 1                                |
| innodb_open_files               | 300                              |
| innodb_rollback_on_timeout      | OFF                              |
| innodb_support_xa               | ON                               |
| innodb_sync_spin_loops          | 20                               |
| innodb_table_locks              | ON                               |
| innodb_thread_concurrency       | 8                                |
| innodb_thread_sleep_delay       | 10000                            |
| interactive_timeout             | 28800                            |
| join_buffer_size                | 131072                           |
| key_buffer_size                 | 402653184                        |
| key_cache_age_threshold         | 300                              |
| key_cache_block_size            | 1024                             |
| key_cache_division_limit        | 100                              |
| language                        | /usr/local/share/mysql/english/  |
| large_files_support             | ON                               |
| large_page_size                 | 0                                |
| large_pages                     | OFF                              |
| lc_time_names                   | en_US                            |
| license                         | GPL                              |
| local_infile                    | ON                               |
| locked_in_memory                | OFF                              |
| log                             | OFF                              |
| log_bin                         | ON                               |
| log_bin_trust_function_creators | OFF                              |
| log_error                       |                                  |
| log_queries_not_using_indexes   | OFF                              |
| log_slave_updates               | OFF                              |
| log_slow_queries                | ON                               |
| log_warnings                    | 1                                |
| long_query_time                 | 10                               |
| low_priority_updates            | OFF                              |
| lower_case_file_system          | OFF                              |
| lower_case_table_names          | 0                                |
| max_allowed_packet              | 1047552                          |
| max_binlog_cache_size           | 4294967295                       |
| max_binlog_size                 | 104857600                        |
| max_connect_errors              | 10                               |
| max_connections                 | 100                              |
| max_delayed_threads             | 20                               |
| max_error_count                 | 64                               |
| max_heap_table_size             | 134217728                        |
| max_insert_delayed_threads      | 20                               |
| max_join_size                   | 4294967295                       |
| max_length_for_sort_data        | 1024                             |
| max_prepared_stmt_count         | 16382                            |
| max_relay_log_size              | 0                                |
| max_seeks_for_key               | 4294967295                       |
| max_sort_length                 | 1024                             |
| max_sp_recursion_depth          | 0                                |
| max_tmp_tables                  | 32                               |
| max_user_connections            | 0                                |
| max_write_lock_count            | 4294967295                       |
| multi_range_count               | 256                              |
| myisam_data_pointer_size        | 6                                |
| myisam_max_sort_file_size       | 2147483647                       |
| myisam_recover_options          | OFF                              |
| myisam_repair_threads           | 1                                |
| myisam_sort_buffer_size         | 67108864                         |
| myisam_stats_method             | nulls_unequal                    |
| net_buffer_length               | 16384                            |
| net_read_timeout                | 30                               |
| net_retry_count                 | 10                               |
| net_write_timeout               | 60                               |
| new                             | OFF                              |
| old_passwords                   | OFF                              |
| open_files_limit                | 4096                             |
| optimizer_prune_level           | 1                                |
| optimizer_search_depth          | 62                               |
| pid_file                        | /usr2/mysql/var/run/mysqld.pid   |
| port                            | 3306                             |
| preload_buffer_size             | 32768                            |
| profiling                       | OFF                              |
| profiling_history_size          | 15                               |
| protocol_version                | 10                               |
| query_alloc_block_size          | 8192                             |
| query_cache_limit               | 1048576                          |
| query_cache_min_res_unit        | 4096                             |
| query_cache_size                | 33554432                         |
| query_cache_type                | ON                               |
| query_cache_wlock_invalidate    | OFF                              |
| query_prealloc_size             | 8192                             |
| range_alloc_block_size          | 2048                             |
| read_buffer_size                | 2093056                          |
| read_only                       | OFF                              |
| read_rnd_buffer_size            | 8384512                          |
| relay_log_purge                 | ON                               |
| relay_log_space_limit           | 0                                |
| rpl_recovery_rank               | 0                                |
| secure_auth                     | OFF                              |
| secure_file_priv                |                                  |
| server_id                       | 2                                |
| skip_external_locking           | ON                               |
| skip_networking                 | OFF                              |
| skip_show_database              | OFF                              |
| slave_compressed_protocol       | OFF                              |
| slave_load_tmpdir               | /usr2/mysql/tmp/                 |
| slave_net_timeout               | 3600                             |
| slave_skip_errors               | ALL                              |
| slave_transaction_retries       | 10                               |
| slow_launch_time                | 2                                |
| socket                          | /usr2/mysql/var/run/mysqld.sock  |
| sort_buffer_size                | 2097144                          |
| sql_big_selects                 | ON                               |
| sql_mode                        |                                  |
| sql_notes                       | ON                               |
| sql_warnings                    | OFF                              |
| ssl_ca                          |                                  |
| ssl_capath                      |                                  |
| ssl_cert                        |                                  |
| ssl_cipher                      |                                  |
| ssl_key                         |                                  |
| storage_engine                  | MyISAM                           |
| sync_binlog                     | 0                                |
| sync_frm                        | ON                               |
| system_time_zone                | CET                              |
| table_cache                     | 512                              |
| table_lock_wait_timeout         | 50                               |
| table_type                      | MyISAM                           |
| thread_cache_size               | 8                                |
| thread_stack                    | 196608                           |
| time_format                     | %H:%i:%s                         |
| time_zone                       | SYSTEM                           |
| timed_mutexes                   | OFF                              |
| tmp_table_size                  | 134217728                        |
| tmpdir                          | /usr2/mysql/tmp/                 |
| transaction_alloc_block_size    | 8192                             |
| transaction_prealloc_size       | 4096                             |
| tx_isolation                    | REPEATABLE-READ                  |
| updatable_views_with_limit      | YES                              |
| version                         | 5.0.45-log                       |
| version_comment                 | Source distribution              |
| version_compile_machine         | i686                             |
| version_compile_os              | pc-linux-gnu                     |
| wait_timeout                    | 28800                            |

ids_stat架构:

SHOW CREATE TABLE ids_stat;
CREATE TABLE `ids_stat` (
  `id_rubrique` bigint(20) NOT NULL default '0',
  `id_stat` int(10) default NULL,
  `id_appli` smallint(5) unsigned NOT NULL default '0',
  `nom_rubrique` varchar(240) NOT NULL,
  `date_creation` timestamp NOT NULL default CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id_rubrique`),
  KEY `id_appli_index` (`id_appli`),
  KEY `date_creation_index` (`date_creation`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 

SRV0示例架构:

CREATE TABLE `SRV201606` (
  `id_rubrique` bigint(20) NOT NULL default '0',
  `id_plaque` tinyint(3) unsigned NOT NULL default '0',
  `id_combinaison` smallint(5) unsigned NOT NULL default '0',
  `_date` date NOT NULL default '0000-00-00',
  `heure` tinyint(3) NOT NULL default '0',
  `minute` tinyint(3) NOT NULL default '0',
  `nombre_entree` mediumint(8) unsigned default '0',
  `nombre_st` mediumint(8) unsigned default NULL,
  `duree_st` mediumint(8) unsigned default NULL,
  `flag` tinyint(3) unsigned NOT NULL default '0',
  PRIMARY KEY  (`_date`,`heure`,`minute`,`id_plaque`,`id_rubrique`,`id_combinaison`),
  KEY `id_rubrique` (`id_rubrique`),
  KEY `id_combinaison` (`id_combinaison`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

4 个答案:

答案 0 :(得分:0)

这不是我完成的答案!

但是你只能在你的大表上运行这个查询并给我执行时间和它的EXPLAIN。

ALTER TABLE SRV201606 ADD KEY idx_rubrique_date(_dateid_rubrique);

SELECT
    i.*,    
    SUM(srv0.duree_st) AS dureeAppelTotale,    
    srv0._date AS dateI,    
    DATE_FORMAT(srv0._date,'%d/%m') AS datekey,    
    DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,    
    heure,    
    SUM(srv0.nombre_st) AS nbAppel 
FROM (
  SELECT    
        id_rubrique
        , id_stat
  FROM    ids_stat
  WHERE    
    id_appli = 24071    
  GROUP BY    id_stat
) AS i
LEFT JOIN SRV201606 srv0 ON i.id_rubrique = srv0.id_rubrique
WHERE
  srv0._date >= '2016-06-01'    
AND 
  srv0._date <= '2016-07-01' 
GROUP BY    id_stat, _date, heure  
ORDER BY _date;

答案 1 :(得分:0)

您可以尝试将_date添加到

CREATE TABLE `SRV201606` (
...

KEY `id_rubrique` (`id_rubrique`),

所以

KEY `id_rubrique` (`_date`,`id_rubrique`),

或者

KEY `date-id_rubrique` (`_date`,`id_rubrique`),

答案 2 :(得分:0)

您可以尝试从日期中删除时间并发布EXPLAIN,如下所示: -

SELECT    
    i.id_stat as idStat,    
    SUM(srv0.duree_st) AS dureeAppelTotale,    
    _date as dateI,    
    DATE_FORMAT(srv0._date,'%d/%m') AS datekey,    
    DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,    
    heure,    
    SUM(srv0.nombre_st) AS nbAppel 
FROM    ids_stat i
INNER JOIN SRV201606 srv0  
ON    i.id_rubrique = srv0.id_rubrique    
WHERE i.id_appli = 24071    
AND srv0._date BETWEEN '2016-06-01' AND '2016-07-01'  
GROUP BY    id_stat, 
            datekeyGroup, 
            heure  
ORDER BY _date

我怀疑MySQL正在尝试将日期转换为返回的每一行的日期/时间。

答案 3 :(得分:0)

ContentPage.BindingContext

你想要两个中午吗?建议这样做:

WHERE i.id_rubrique = srv0.id_rubrique    
  AND i.id_appli = 24071    
  AND srv0._date >= '2016-06-01 00:00:00'    
  AND srv0._date <= '2016-07-01 00:00:00'  

需要这些索引以便于从 AND srv0._date >= '2016-06-01' AND srv0._date < '2016-06-01' + INTERVAL 1 MONTH 开始:

i

需要这些索引以便于从i: (id_appli, id_rubrique, id_stat) -- in this order; and is 'covering' srv0: (id_rubrique, _date) -- '=' part first, then 'range' 开始:

srv0

添加所有这些索引将为优化器提供从任一表开始并有效执行的选项。