如何在MySQL中避免“使用keycache修复”?

时间:2011-06-05 16:00:41

标签: mysql

这真令人抓狂。我已经按照我在互联网上找到的设置的每条指令进行了操作,我无法理解这一点。

基本上,我有一个大约800万行的表。我需要像这样创建这个表的备份:

create table mytable_backup like mytable

我的生产服务器需要几个小时,这是一个运行在EngineYard上的Amazon EC2实例。我的MacBook Pro只需几分钟。这是MySQL在后台做的另一件令人讨厌的事情,你无法猜测它是如何决定做一些如此愚蠢的事情。

顺便说一下,tmp目录中有超过330G可用,所以这不是问题。

但这是“free -m”产生的结果:

deploy@domU-12-31-39-02-35-31 ~ $ free -m
             total       used       free     shared    buffers     cached
Mem:          1740       1728         11          0         14       1354
-/+ buffers/cache:        359       1380
Swap:          895          2        893

我不知道如何阅读,但免费栏下的“11”看起来不太好。

我正在跑步:

Server version: 5.0.51-log Gentoo Linux mysql-community-5.0.51

这是我的配置文件:

# /etc/mysql/my.cnf: The global mysql configuration file.
# $Header: /var/cvsroot/gentoo-x86/dev-db/mysql/files/my.cnf-4.1,v 1.3 2006/05/05 19:51:40 chtekk Exp $

# The following options will be passed to all MySQL clients
[client]
port                = 3306

[mysql]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8

[mysqladmin]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8

[mysqlcheck]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8

[mysqldump]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8

[mysqlimport]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8

[mysqlshow]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8

[myisamchk]
character-sets-dir=/usr/share/mysql/charsets

[myisampack]
character-sets-dir=/usr/share/mysql/charsets


[mysqld_safe]
err-log             = /db/mysql/log/mysql.err

# To allow table cache to be raised
open-file-limit = 4096

[mysqld]
max_connections         = 300
innodb_file_per_table       = 1

log-slow-queries        = /db/mysql/log/slow_query.log
long_query_time         = 2000000
ft_min_word_len         = 3

max_heap_table_size         = 64M
tmp_table_size          = 64M

server-id           = 1
log-bin             = /db/mysql/master-bin
log-bin-index           = /db/mysql/master-bin.index
# END master/slave configuration

character-set-server        = utf8
default-character-set       = utf8
user                = mysql
port                = 3306
socket                  = /var/run/mysqld/mysqld.sock
pid-file            = /var/run/mysqld/mysqld.pid
log-error           = /db/mysql/log/mysqld.err
basedir             = /usr
datadir             = /db/mysql

key_buffer          = 32M
max_allowed_packet      = 32M
table_cache         = 1024
thread_cache            = 512
sort_buffer_size        = 100M
net_buffer_length       = 64K
read_buffer_size        = 1M
read_rnd_buffer_size        = 1M
myisam_sort_buffer_size     = 100M
myisam_max_sort_file_size   = 2G
myisam_repair_threads           = 1
language            = /usr/share/mysql/english

# security:
# using "localhost" in connects uses sockets by default
# skip-networking
# bind-address          = 127.0.0.1

# point the following paths to different dedicated disks
tmpdir              = /mnt/mysql/tmp
# log-update            = /path-to-dedicated-directory/hostname

# you need the debug USE flag enabled to use the following directives,
# if needed, uncomment them, start the server and issue 
# #tail -f /tmp/mysqld.sql /tmp/mysqld.trace
# this will show you *exactly* what's happening in your server ;)

#log                = /tmp/mysqld.sql
#gdb
#debug              = d:t:i:o,/tmp/mysqld.trace
#one-thread


# the rest of the innodb config follows:
# don't eat too much memory, we're trying to be safe on 64Mb boxes
# you might want to bump this up a bit on boxes with more RAM
innodb_buffer_pool_size     = 1275M
# this is the default, increase it if you have lots of tables
innodb_additional_mem_pool_size = 16M
#
# i'd like to use /var/lib/mysql/innodb, but that is seen as a database :-(
# and upstream wants things to be under /var/lib/mysql/, so that's the route
# we have to take for the moment
#innodb_data_home_dir       = /var/lib/mysql/
#innodb_log_arch_dir        = /var/lib/mysql/
#innodb_log_group_home_dir  = /var/lib/mysql/
# you may wish to change this size to be more suitable for your system
# the max is there to avoid run-away growth on your machine
innodb_data_file_path       = ibdata1:20M:autoextend

# we keep this at around 25% of of innodb_buffer_pool_size
# sensible values range from 1MB to (1/innodb_log_files_in_group*innodb_buffer_pool_size)
innodb_log_file_size        = 96M

# this is the default, increase it if you have very large transactions going on
innodb_log_buffer_size      = 8M

# this is the default and won't hurt you
# you shouldn't need to tweak it
innodb_log_files_in_group   = 2

# see the innodb config docs, the other options are not always safe
# This is not good for performance when used with bin_sync.  Disabling.
innodb_flush_log_at_trx_commit  = 2
innodb_flush_method     = O_DIRECT
innodb_lock_wait_timeout    = 50

query_cache_size        = 16M
query_cache_type        = 1

[mysqldump]
quick
max_allowed_packet      = 16M

[mysql]
# uncomment the next directive if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer          = 20M
sort_buffer_size        = 20M
read_buffer         = 2M
write_buffer            = 2M

[myisamchk]
key_buffer          = 20M
sort_buffer_size        = 20M
read_buffer         = 2M
write_buffer            = 2M
ft_min_word_len         = 3

[mysqlhotcopy]
interactive-timeout

3 个答案:

答案 0 :(得分:2)

对于它的价值,11 megs free是完全没问题的。这是11兆的内存没有被用于任何东西,并且就硬件而言“浪费”了。实数是缓存中使用的“1380”,加上未使用的11兆。如有必要,可以吹走高速缓存。

您的系统有近1400 MB的RAM。

答案 1 :(得分:1)

您必须查看 myisam_max_sort_file_size myisam_sort_buffer_size

的设置

如果所有键的总和小于myisam_max_sort_file_size,那么在最坏的情况下,排序将在MyISAM表中着陆,这是一件好事。

否则,它将恢复为keycache。这意味着将必要的.MYI索引页加载到密钥缓存中并遍历内存中的索引页。没人想要!!!!

此变量的当前设置为2G。

查看正在构建的密钥。添加它们。如果所有密钥大小的总和超过2G,那么keycache一路!!!你将不得不提高这个价值。您可以使用

将会话值增加到4G
SET myisam_max_sort_file_size = 1024 * 1024 * 1024 * 4;
SET myisam_sort_buffer_size = 1024 * 1024 * 1024 * 4;

或者您可以像这样直接种植数字:

SET myisam_max_sort_file_size = 4294967296;
SET myisam_sort_buffer_size = 4294967296;

在执行ENABLE KEYS之前;

如果您只是对备份数据感兴趣,为什么要将其开始索引?尝试使用 ARCHIVE 存储引擎。它没有任何索引。执行以下操作:

CREATE TABLE mytable_backup LIKE mytable;
ALTER TABLE mytable_backup ENGINE=ARCHIVE;
INSERT INTO mytable_backup SELECT * FROM mytable;

我还注意到您使用的是Amazon EC2。我之前从未去过EC2。运行此命令:

 SHOW ENGINES;

+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine     | Support | Comment                                                        | Transactions | XA   | Savepoints |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| InnoDB     | DEFAULT | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
| MRG_MYISAM | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| BLACKHOLE  | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| CSV        | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| MEMORY     | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| FEDERATED  | YES     | Federated MySQL storage engine                                 | NO           | NO   | NO         |
| ARCHIVE    | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| MyISAM     | YES     | Default engine as of MySQL 3.23 with great performance         | NO           | NO   | NO         |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+

如果ARCHIVE存储引擎出现在列表中且支持为是,则可以选择备份到ARCHIVE表。如果没有,则必须调整myisam_max_sort_file_size和myisam_sort_buffer_size。

答案 2 :(得分:1)

你可以尝试

create table backup_table as (select * from production table) engine=myisam

这应该创建只包含数据而不包含任何键的表。然后,您可以通过执行

来添加密钥
alter table backup_table add index(column_name)

我已经成功完成了几次,并且通常比使用钥匙插入速度快2倍。