基本上,我们有一个很大的生产问题,我们的sql查询在结束前需要超过300秒。
我们使用Mysql作为SQL服务器,我们的表使用MyISAM。
有问题的查询是两个表之间的基本查询,以及排序和分组。问题是每个表都有超过3000万行。
以下是此查询的错误示例:
SELECT
i.id_stat as idStat,
SUM(srv0.duree_st) AS dureeAppelTotale,
_date as dateI,
DATE_FORMAT(srv0._date,'%d/%m') AS datekey,
DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,
heure,
SUM(srv0.nombre_st) AS nbAppel
FROM ids_stat i , SRV201606 srv0
WHERE i.id_rubrique = srv0.id_rubrique
AND i.id_appli = 24071
AND srv0._date >= '2016-06-01 00:00:00'
AND srv0._date <= '2016-07-01 00:00:00'
GROUP BY id_stat, datekeyGroup, heure
ORDER BY _date
ids_stat表上的索引是:
show index from ids_stat;
+----------+------------+---------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+----------+------------+---------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| ids_stat | 0 | PRIMARY | 1 | id_rubrique | A | 7435537 | NULL | NULL | | BTREE | |
| ids_stat | 1 | id_appli_index | 1 | id_appli | A | 13593 | NULL | NULL | | BTREE | |
| ids_stat | 1 | date_creation_index | 1 | date_creation | A | 2478512 | NULL | NULL | | BTREE | |
SRV0表上的,索引是:
show index from SRV201606;
+-----------+------------+----------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------+------------+----------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+
| SRV201606 | 0 | PRIMARY | 1 | _date | A | NULL | NULL | NULL | | BTREE | |
| SRV201606 | 0 | PRIMARY | 2 | heure | A | NULL | NULL | NULL | | BTREE | |
| SRV201606 | 0 | PRIMARY | 3 | minute | A | NULL | NULL | NULL | | BTREE | |
| SRV201606 | 0 | PRIMARY | 4 | id_plaque | A | NULL | NULL | NULL | | BTREE | |
| SRV201606 | 0 | PRIMARY | 5 | id_rubrique | A | NULL | NULL | NULL | | BTREE | |
| SRV201606 | 0 | PRIMARY | 6 | id_combinaison | A | 27472765 | NULL | NULL | | BTREE | |
| SRV201606 | 1 | id_rubrique | 1 | id_rubrique | A | NULL | NULL | NULL | | BTREE | |
| SRV201606 | 1 | id_combinaison | 1 | id_combinaison | A | NULL | NULL | NULL | | BTREE | |
此外,这是对查询的解释:
+----+-------------+-------+------+------------------------+----------------+---------+-----------------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+------------------------+----------------+---------+-----------------------------+------+---------------------------------+
| 1 | SIMPLE | i | ref | PRIMARY,id_appli_index | id_appli_index | 2 | const | 153 | Using temporary; Using filesort |
| 1 | SIMPLE | srv0 | ref | PRIMARY,id_rubrique | id_rubrique | 8 | StatVoxProdV2.i.id_rubrique | 16 | Using where |
我们还读到这些旧的连接在哪里是无害的,因为它们被sql引擎转录为内部连接。
你能帮我们一些方法调整我们的mysql服务器,以便它可以提升吗?有没有可以帮助我们的配置?
非常感谢
[编辑] 正如评论中所说,这是他的版本:
SELECT VERSION();
+------------+
| VERSION() |
+------------+
| 5.0.45-log |
服务器变量:
show Variables;
+---------------------------------+----------------------------------+
| Variable_name | Value |
+---------------------------------+----------------------------------+
| auto_increment_increment | 1 |
| auto_increment_offset | 1 |
| automatic_sp_privileges | ON |
| back_log | 50 |
| basedir | /usr/local/ |
| binlog_cache_size | 32768 |
| bulk_insert_buffer_size | 8388608 |
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | latin1 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/local/share/mysql/charsets/ |
| collation_connection | utf8_general_ci |
| collation_database | latin1_swedish_ci |
| collation_server | latin1_swedish_ci |
| completion_type | 0 |
| concurrent_insert | 1 |
| connect_timeout | 5 |
| datadir | THEDATADIR |
| date_format | %Y-%m-%d |
| datetime_format | %Y-%m-%d %H:%i:%s |
| default_week_format | 0 |
| delay_key_write | ON |
| delayed_insert_limit | 100 |
| delayed_insert_timeout | 300 |
| delayed_queue_size | 1000 |
| div_precision_increment | 4 |
| engine_condition_pushdown | OFF |
| expire_logs_days | 32 |
| flush | OFF |
| flush_time | 0 |
| ft_boolean_syntax | + -><()~*:""&| |
| ft_max_word_len | 84 |
| ft_min_word_len | 4 |
| ft_query_expansion_limit | 20 |
| ft_stopword_file | (built-in) |
| group_concat_max_len | 1024 |
| have_archive | NO |
| have_bdb | NO |
| have_blackhole_engine | NO |
| have_compress | YES |
| have_crypt | YES |
| have_csv | NO |
| have_dynamic_loading | YES |
| have_example_engine | NO |
| have_federated_engine | NO |
| have_geometry | YES |
| have_innodb | DISABLED |
| have_isam | NO |
| have_merge_engine | YES |
| have_ndbcluster | NO |
| have_openssl | NO |
| have_ssl | NO |
| have_query_cache | YES |
| have_raid | NO |
| have_rtree_keys | YES |
| have_symlink | YES |
| hostname | THEHOSTNAME |
| init_connect | |
| init_file | |
| init_slave | |
| innodb_additional_mem_pool_size | 1048576 |
| innodb_autoextend_increment | 8 |
| innodb_buffer_pool_awe_mem_mb | 0 |
| innodb_buffer_pool_size | 1048576 |
| innodb_checksums | ON |
| innodb_commit_concurrency | 0 |
| innodb_concurrency_tickets | 500 |
| innodb_data_file_path | |
| innodb_data_home_dir | |
| innodb_doublewrite | ON |
| innodb_fast_shutdown | 1 |
| innodb_file_io_threads | 4 |
| innodb_file_per_table | OFF |
| innodb_flush_log_at_trx_commit | 1 |
| innodb_flush_method | |
| innodb_force_recovery | 0 |
| innodb_lock_wait_timeout | 50 |
| innodb_locks_unsafe_for_binlog | OFF |
| innodb_log_arch_dir | |
| innodb_log_archive | OFF |
| innodb_log_buffer_size | 1048576 |
| innodb_log_file_size | 5242880 |
| innodb_log_files_in_group | 2 |
| innodb_log_group_home_dir | |
| innodb_max_dirty_pages_pct | 90 |
| innodb_max_purge_lag | 0 |
| innodb_mirrored_log_groups | 1 |
| innodb_open_files | 300 |
| innodb_rollback_on_timeout | OFF |
| innodb_support_xa | ON |
| innodb_sync_spin_loops | 20 |
| innodb_table_locks | ON |
| innodb_thread_concurrency | 8 |
| innodb_thread_sleep_delay | 10000 |
| interactive_timeout | 28800 |
| join_buffer_size | 131072 |
| key_buffer_size | 402653184 |
| key_cache_age_threshold | 300 |
| key_cache_block_size | 1024 |
| key_cache_division_limit | 100 |
| language | /usr/local/share/mysql/english/ |
| large_files_support | ON |
| large_page_size | 0 |
| large_pages | OFF |
| lc_time_names | en_US |
| license | GPL |
| local_infile | ON |
| locked_in_memory | OFF |
| log | OFF |
| log_bin | ON |
| log_bin_trust_function_creators | OFF |
| log_error | |
| log_queries_not_using_indexes | OFF |
| log_slave_updates | OFF |
| log_slow_queries | ON |
| log_warnings | 1 |
| long_query_time | 10 |
| low_priority_updates | OFF |
| lower_case_file_system | OFF |
| lower_case_table_names | 0 |
| max_allowed_packet | 1047552 |
| max_binlog_cache_size | 4294967295 |
| max_binlog_size | 104857600 |
| max_connect_errors | 10 |
| max_connections | 100 |
| max_delayed_threads | 20 |
| max_error_count | 64 |
| max_heap_table_size | 134217728 |
| max_insert_delayed_threads | 20 |
| max_join_size | 4294967295 |
| max_length_for_sort_data | 1024 |
| max_prepared_stmt_count | 16382 |
| max_relay_log_size | 0 |
| max_seeks_for_key | 4294967295 |
| max_sort_length | 1024 |
| max_sp_recursion_depth | 0 |
| max_tmp_tables | 32 |
| max_user_connections | 0 |
| max_write_lock_count | 4294967295 |
| multi_range_count | 256 |
| myisam_data_pointer_size | 6 |
| myisam_max_sort_file_size | 2147483647 |
| myisam_recover_options | OFF |
| myisam_repair_threads | 1 |
| myisam_sort_buffer_size | 67108864 |
| myisam_stats_method | nulls_unequal |
| net_buffer_length | 16384 |
| net_read_timeout | 30 |
| net_retry_count | 10 |
| net_write_timeout | 60 |
| new | OFF |
| old_passwords | OFF |
| open_files_limit | 4096 |
| optimizer_prune_level | 1 |
| optimizer_search_depth | 62 |
| pid_file | /usr2/mysql/var/run/mysqld.pid |
| port | 3306 |
| preload_buffer_size | 32768 |
| profiling | OFF |
| profiling_history_size | 15 |
| protocol_version | 10 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 33554432 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
| range_alloc_block_size | 2048 |
| read_buffer_size | 2093056 |
| read_only | OFF |
| read_rnd_buffer_size | 8384512 |
| relay_log_purge | ON |
| relay_log_space_limit | 0 |
| rpl_recovery_rank | 0 |
| secure_auth | OFF |
| secure_file_priv | |
| server_id | 2 |
| skip_external_locking | ON |
| skip_networking | OFF |
| skip_show_database | OFF |
| slave_compressed_protocol | OFF |
| slave_load_tmpdir | /usr2/mysql/tmp/ |
| slave_net_timeout | 3600 |
| slave_skip_errors | ALL |
| slave_transaction_retries | 10 |
| slow_launch_time | 2 |
| socket | /usr2/mysql/var/run/mysqld.sock |
| sort_buffer_size | 2097144 |
| sql_big_selects | ON |
| sql_mode | |
| sql_notes | ON |
| sql_warnings | OFF |
| ssl_ca | |
| ssl_capath | |
| ssl_cert | |
| ssl_cipher | |
| ssl_key | |
| storage_engine | MyISAM |
| sync_binlog | 0 |
| sync_frm | ON |
| system_time_zone | CET |
| table_cache | 512 |
| table_lock_wait_timeout | 50 |
| table_type | MyISAM |
| thread_cache_size | 8 |
| thread_stack | 196608 |
| time_format | %H:%i:%s |
| time_zone | SYSTEM |
| timed_mutexes | OFF |
| tmp_table_size | 134217728 |
| tmpdir | /usr2/mysql/tmp/ |
| transaction_alloc_block_size | 8192 |
| transaction_prealloc_size | 4096 |
| tx_isolation | REPEATABLE-READ |
| updatable_views_with_limit | YES |
| version | 5.0.45-log |
| version_comment | Source distribution |
| version_compile_machine | i686 |
| version_compile_os | pc-linux-gnu |
| wait_timeout | 28800 |
ids_stat架构:
SHOW CREATE TABLE ids_stat;
CREATE TABLE `ids_stat` (
`id_rubrique` bigint(20) NOT NULL default '0',
`id_stat` int(10) default NULL,
`id_appli` smallint(5) unsigned NOT NULL default '0',
`nom_rubrique` varchar(240) NOT NULL,
`date_creation` timestamp NOT NULL default CURRENT_TIMESTAMP,
PRIMARY KEY (`id_rubrique`),
KEY `id_appli_index` (`id_appli`),
KEY `date_creation_index` (`date_creation`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
SRV0示例架构:
CREATE TABLE `SRV201606` (
`id_rubrique` bigint(20) NOT NULL default '0',
`id_plaque` tinyint(3) unsigned NOT NULL default '0',
`id_combinaison` smallint(5) unsigned NOT NULL default '0',
`_date` date NOT NULL default '0000-00-00',
`heure` tinyint(3) NOT NULL default '0',
`minute` tinyint(3) NOT NULL default '0',
`nombre_entree` mediumint(8) unsigned default '0',
`nombre_st` mediumint(8) unsigned default NULL,
`duree_st` mediumint(8) unsigned default NULL,
`flag` tinyint(3) unsigned NOT NULL default '0',
PRIMARY KEY (`_date`,`heure`,`minute`,`id_plaque`,`id_rubrique`,`id_combinaison`),
KEY `id_rubrique` (`id_rubrique`),
KEY `id_combinaison` (`id_combinaison`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
答案 0 :(得分:0)
这不是我完成的答案!
但是你只能在你的大表上运行这个查询并给我执行时间和它的EXPLAIN。
ALTER TABLE SRV201606 ADD KEY idx_rubrique_date(_date
,id_rubrique
);
SELECT
i.*,
SUM(srv0.duree_st) AS dureeAppelTotale,
srv0._date AS dateI,
DATE_FORMAT(srv0._date,'%d/%m') AS datekey,
DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,
heure,
SUM(srv0.nombre_st) AS nbAppel
FROM (
SELECT
id_rubrique
, id_stat
FROM ids_stat
WHERE
id_appli = 24071
GROUP BY id_stat
) AS i
LEFT JOIN SRV201606 srv0 ON i.id_rubrique = srv0.id_rubrique
WHERE
srv0._date >= '2016-06-01'
AND
srv0._date <= '2016-07-01'
GROUP BY id_stat, _date, heure
ORDER BY _date;
答案 1 :(得分:0)
您可以尝试将_date添加到
CREATE TABLE `SRV201606` (
...
KEY `id_rubrique` (`id_rubrique`),
所以
KEY `id_rubrique` (`_date`,`id_rubrique`),
或者
KEY `date-id_rubrique` (`_date`,`id_rubrique`),
答案 2 :(得分:0)
您可以尝试从日期中删除时间并发布EXPLAIN,如下所示: -
SELECT
i.id_stat as idStat,
SUM(srv0.duree_st) AS dureeAppelTotale,
_date as dateI,
DATE_FORMAT(srv0._date,'%d/%m') AS datekey,
DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,
heure,
SUM(srv0.nombre_st) AS nbAppel
FROM ids_stat i
INNER JOIN SRV201606 srv0
ON i.id_rubrique = srv0.id_rubrique
WHERE i.id_appli = 24071
AND srv0._date BETWEEN '2016-06-01' AND '2016-07-01'
GROUP BY id_stat,
datekeyGroup,
heure
ORDER BY _date
我怀疑MySQL正在尝试将日期转换为返回的每一行的日期/时间。
答案 3 :(得分:0)
ContentPage.BindingContext
你想要两个中午吗?建议这样做:
WHERE i.id_rubrique = srv0.id_rubrique
AND i.id_appli = 24071
AND srv0._date >= '2016-06-01 00:00:00'
AND srv0._date <= '2016-07-01 00:00:00'
需要这些索引以便于从 AND srv0._date >= '2016-06-01'
AND srv0._date < '2016-06-01' + INTERVAL 1 MONTH
开始:
i
需要这些索引以便于从i: (id_appli, id_rubrique, id_stat) -- in this order; and is 'covering'
srv0: (id_rubrique, _date) -- '=' part first, then 'range'
开始:
srv0
添加所有这些索引将为优化器提供从任一表开始并有效执行的选项。