大数据的Mysql性能

时间:2015-04-15 06:49:51

标签: mysql

嘿,我有1.25亿个数据,但是mysql查询速度很慢。

这是my.ini配置

    # The following options will be passed to all MySQL clients
[client]
#password   = your_password
port        = 3306
socket      = /tmp/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[wampmysqld]
port        = 3306
socket      = /tmp/mysql.sock
key_buffer_size = 16M
max_allowed_packet = 1M
sort_buffer_size = 512M
net_buffer_length = 8K
read_buffer_size = 256M
read_rnd_buffer_size = 128M
myisam_sort_buffer_size = 64M
basedir=c:/wamp/bin/mysql/mysql5.6.17
log-error=c:/wamp/logs/mysql.log
datadir=c:/wamp/bin/mysql/mysql5.6.17/data

lc-messages-dir=c:/wamp/bin/mysql/mysql5.6.17/share

# Change your locale here !
lc-messages=fr_FR

# Avoid warning
explicit_defaults_for_timestamp = TRUE

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
# 
#skip-networking

# Disable Federated by default
skip-federated

# Replication Master Server (default)
# binary logging is required for replication
log-bin=mysql-bin

# binary logging format - mixed recommended
binlog_format=mixed

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id   = 1

# Replication Slave (comment out master section to use this)


# New for MySQL 5.6 if no slave
skip-slave-start

#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
#    the syntax is:
#
#    CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
#    MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
#    where you replace <host>, <user>, <password> by quoted strings and
#    <port> by the master's port number (3306 by default).
#
#    Example:
#
#    CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
#    MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
#    start replication for the first time (even unsuccessfully, for example
#    if you mistyped the password in master-password and the slave fails to
#    connect), the slave will create a master.info file, and any later
#    change in this file to the variables' values below will be ignored and
#    overridden by the content of the master.info file, unless you shutdown
#    the slave server, delete master.info and restart the slaver server.
#    For that reason, you may want to leave the lines below untouched
#    (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id       = 2
#
# The replication master for this slave - required
#master-host     =   <hostname>
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user     =   <username>
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password =   <password>
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port     =  <port>
#
# binary logging - not required for slaves, but recommended
#log-bin=mysql-bin

# Point the following paths to different dedicated disks
#tmpdir     = /tmp/     
#log-update     = /path-to-dedicated-directory/hostname

# Uncomment the following if you are using InnoDB tables
innodb_data_home_dir = "c:/wamp/bin/mysql/mysql5.6.17/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "c:/wamp/bin/mysql/mysql5.6.17/data"
#innodb_log_arch_dir = C:\mysql\data/

# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 12G
innodb_additional_mem_pool_size = 256M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 512M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 200

[mysqldump]
quick
max_allowed_packet = 64M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer = 64M
sort_buffer_size = 128M
read_buffer = 128M
write_buffer = 10M

[myisamchk]
key_buffer = 64M
sort_buffer_size = 128M
read_buffer = 128M
write_buffer = 10M

[mysqlhotcopy]
interactive-timeout

[mysqld]
port=3306
explicit_defaults_for_timestamp = TRUE

innodb_data_home_dir = "c:/wamp/bin/mysql/mysql5.6.17/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "c:/wamp/bin/mysql/mysql5.6.17/data"
#innodb_log_arch_dir = C:\mysql\data/

# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 12G
innodb_additional_mem_pool_size = 256M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 512M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 200
query_cache_size=10M
key_buffer_size=512M
join_buffer_size=256M
thread_cache_size=512
tmp_table_size=256M
read_buffer_size = 1G
read_rnd_buffer_size = 1G
innodb_read_io_threads=4

这是我的示例查询

SELECT 
    wp.wp_id as wp_id,
    wpt.wp_template_name as wp_template,
    wp.project_id as project_id,
    wp.site_id as site_id,
    s.site_name as site_name,
    s.area as area,
    cus.customer_name as customer,
    ven.vendor_name as vendor,
    wp.status as status,
    wp.target_date as target_date,
    wp.remark as remark,
    wp.contract_id as vendor_contract,
    wp.po_id as customer_po_id
FROM wp as wp 
LEFT JOIN wp_template as wpt ON wp.wp_template_id=wpt.wp_template_id
LEFT JOIN site as s ON wp.site_id=s.site_id 
LEFT JOIN customer_po as cpo ON wp.po_id=cpo.po_id
LEFT JOIN customer as cus ON cpo.customer_id=cus.customer_id
LEFT JOIN vendor as ven ON wp.vendor_id=ven.vendor_id
WHERE 1=1 and wp.project_id='project_2'

解释此查询

SIMPLE  wp  ref project_id  project_id  203 const   2122262 Using index condition
SIMPLE  wpt ALL wp_template_id              Using where; Using join buffer (Block Nested Loop)
SIMPLE  s   ref site_id site_id 203 kin_pmis.wp.site_id 
SIMPLE  cpo ref idx_customer_po_po_id   idx_customer_po_po_id   203 kin_pmis.wp.po_id   
SIMPLE  cus ALL customer_id             Using where; Using join buffer (Block Nested Loop)
SIMPLE  ven ALL vendor_id               Using where; Using join buffer (Block Nested Loop)

我将innodb_buffer_pool_size设置为12G,但在我的内存中只使用了大约6G,计数查询可能需要很长时间。

1 个答案:

答案 0 :(得分:0)

没有这些表的架构它有点难,但我的第一个建议是你可能会丢失所有join子句的一些索引(你有wpt.wp_template_id索引,s.site_id ,cpo.po_id,...)

同样从解释中看来,无论你做什么,这都可能不是一个快速的查询,因为你的数据库中有100米的记录。