通过大量连接提高MySQL查询速度

时间:2014-08-12 08:05:04

标签: mysql left-join database-performance

我在MySQL中有一个使用多个连接的查询,此时它运行缓慢 - 平均需要大约35秒才能运行。

查询是:

SELECT t.id,
       CASE t.emp_accepted
           WHEN '1' THEN 'No'
           WHEN '0' THEN 'Yes'
       END AS accepted,
       e.department,
       e.works_id,
       e.first_name,
       e.sur_name,
       e.job_title,
       e.job_status,
       e.site_id,
       e.manager,
       d1.department_name AS dept_name,
       d2.department_name AS sub_dept_name,
       temp_hours_worked.hours AS hours,
       s.office_name AS site_name,
       CONCAT(e2.first_name, ' ', e2.sur_name) AS manager_name,
       CONCAT(e3.first_name, ' ', e3.sur_name) AS validated_by
FROM time t
LEFT JOIN employee e
    ON t.employee_id = e.employee_id
LEFT JOIN departments d1
    ON e.department = d1.id
LEFT JOIN departments d2
    ON e.sub_department = d2.id
LEFT JOIN site s
    ON e.site_id = s.id
LEFT JOIN employee e2
    ON e.manager = e2.id
LEFT JOIN employee e3
    ON t.manager_id = e3.id
LEFT JOIN temp_hours_worked
    ON temp_hours_worked.week_beginning = t.week_beginning
        AND temp_hours_worked.employee_id = t.employee_id
        AND temp_hours_worked.company_id=?
WHERE t.company_id = ?;

说明:

+----+-------------+-------------------+--------+---------------+-------------+---------+-----------------------------------------+------+-------+
| id | select_type | table             | type   | possible_keys | key         | key_len | ref                                     | rows | Extra |
+----+-------------+-------------------+--------+---------------+-------------+---------+-----------------------------------------+------+-------+
|  1 | SIMPLE      | t                 | ref    | company_id    | company_id  | 4       | const                                   | 5566 |       |
|  1 | SIMPLE      | e                 | ref    | employee_id   | employee_id | 4       | DBNAME.t.employee_id                    |    1 |       |
|  1 | SIMPLE      | d1                | eq_ref | PRIMARY       | PRIMARY     | 4       | DBNAME.e.department                     |    1 |       |
|  1 | SIMPLE      | d2                | eq_ref | PRIMARY       | PRIMARY     | 4       | DBNAME.e.sub_department                 |    1 |       |
|  1 | SIMPLE      | s                 | eq_ref | PRIMARY       | PRIMARY     | 4       | DBNAME.e.site_id                        |    1 |       |
|  1 | SIMPLE      | e2                | eq_ref | PRIMARY       | PRIMARY     | 4       | DBNAME.e.manager                        |    1 |       |
|  1 | SIMPLE      | e3                | eq_ref | PRIMARY       | PRIMARY     | 4       | DBNAME.t.manager_id                     |    1 |       |
|  1 | SIMPLE      | temp_hours_worked | ref    | company_id    | company_id  | 4       | const                                   | 5566 |       |
+----+-------------+-------------------+--------+---------------+-------------+---------+-----------------------------------------+------+-------+

MySQL版本是在Centos 6.5 64位上运行的5.5.31,服务器是8核,4GB RAM和SSD磁盘。盒子上的平均负载是:

load average: 0.24, 0.29, 0.29

和空闲记忆显示为:

             total       used       free     shared    buffers     cached
Mem:          3880       3067        813          0        177       1065
-/+ buffers/cache:       1825       2055
Swap:         1023          0       1023

磁盘空间正常:

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       45G   11G   32G  26% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
/usr/tmpDSK    1008M   51M  907M   6% /tmp

hdparm -Tt / dev / xvda1

的输出
 Timing cached reads:   12538 MB in  1.99 seconds = 6297.76 MB/sec
 Timing buffered disk reads: 826 MB in  3.00 seconds = 275.27 MB/sec

的my.cnf:

[mysql]

# CLIENT #
port                           = 3306
socket                         = /var/lib/mysql/mysql.sock

[mysqld]


local-infile=0

# GENERAL #
user                           = mysql
default_storage_engine         = InnoDB
socket                         = /var/lib/mysql/mysql.sock
pid_file                       = /var/lib/mysql/mysql.pid

# MyISAM #
key_buffer_size                = 32M
myisam_recover                 = FORCE,BACKUP

# SAFETY #
max_allowed_packet             = 16M
max_connect_errors             = 1000000
skip_name_resolve
#sql_mode                      = NO_AUTO_CREATE_USER,NO_AUTO_VALUE_ON_ZERO,NO_ENGINE_SUBSTITUTION,NO_ZERO_DATE,NO_ZERO_IN_DATE,ONLY_FULL_GROUP_BY
sysdate_is_now                 = 1
innodb                         = FORCE
#innodb_strict_mode            = 1

# DATA STORAGE #
datadir                        = /var/lib/mysql/

# BINARY LOGGING #
log_bin                        = /var/lib/mysql/mysql-bin
expire_logs_days               = 14
sync_binlog                    = 1

# CACHES AND LIMITS #
tmp_table_size                 = 32M
max_heap_table_size            = 32M
query_cache_type               = 0
query_cache_size               = 0

max_connections                = 500
thread_cache_size              = 50
open_files_limit               = 65535
table_definition_cache         = 4096
table_open_cache               = 4096

# INNODB #
innodb_flush_method            = O_DIRECT
innodb_log_files_in_group      = 2
innodb_log_file_size           = 128M
innodb_flush_log_at_trx_commit = 1
innodb_file_per_table          = 1
innodb_buffer_pool_size        = 1456M

# LOGGING #
log_error                      = /var/lib/mysql/mysql-error.log
#log_queries_not_using_indexes  = 1
slow_query_log                 = 1
slow_query_log_file            = /var/lib/mysql/mysql-slow.log

被查询/加入的列都是必需的,无法删除,我发现很多列上没有索引,但由于它们只是单行,我不确定是否重要 - 我还能做什么吗如何加快查询速度?

1 个答案:

答案 0 :(得分:3)

这些查询永远不会在那些硬件规范上变得那么慢。 explain输出表示所有已连接的字段使用非常优化的索引,并且仅扫描仅5566行。唯一的索引改进可能是temp_hours_worked字段week_beginning, employee_id, company_id上的综合索引,但这种情况永远不会有太大差别。根据解释输出,甚至没有任何filesorts或临时表。

我怀疑你是否遇到了锁定问题(你显示的负载很低,但是并没有告诉同一个查询在这些相同的表上运行了多少),或者你的MySQL因配置不足而功能不足(使用默认的tiny.config设置或类似设置。

要检查的事项:

  • 使用hdparm -Tt /dev/sdX来测试硬盘性能 - 可能会出现SSD磁盘或RAID阵列
  • 检查您的效果设置。不要犹豫,将buffer中的所有my.cnf设置至少设置为当前值的两倍,您需要备用RAM。少数可能需要更高的设置。像MySQLTuner这样的脚本可能对此有所帮助。

同时检查问题是否可在其他服务器上重现。

启动MySQL缓冲区值的良好开端是将此位添加到my.cnf

key_buffer = 768M
table_cache = 1024
sort_buffer_size = 4M
read_buffer_size = 4M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 128M
query_cache_size = 128M
thread_concurrency = 16
table_open_cache = 2048
tmp_table_size = 64M
max_heap_table_size = 64M

您可以在phpMyAdmin(服务器 - >变量)中查看当前值。