这真令人抓狂。我已经按照我在互联网上找到的设置的每条指令进行了操作,我无法理解这一点。
基本上,我有一个大约800万行的表。我需要像这样创建这个表的备份:
create table mytable_backup like mytable
我的生产服务器需要几个小时,这是一个运行在EngineYard上的Amazon EC2实例。我的MacBook Pro只需几分钟。这是MySQL在后台做的另一件令人讨厌的事情,你无法猜测它是如何决定做一些如此愚蠢的事情。
顺便说一下,tmp目录中有超过330G可用,所以这不是问题。
但这是“free -m”产生的结果:
deploy@domU-12-31-39-02-35-31 ~ $ free -m
total used free shared buffers cached
Mem: 1740 1728 11 0 14 1354
-/+ buffers/cache: 359 1380
Swap: 895 2 893
我不知道如何阅读,但免费栏下的“11”看起来不太好。
我正在跑步:
Server version: 5.0.51-log Gentoo Linux mysql-community-5.0.51
这是我的配置文件:
# /etc/mysql/my.cnf: The global mysql configuration file.
# $Header: /var/cvsroot/gentoo-x86/dev-db/mysql/files/my.cnf-4.1,v 1.3 2006/05/05 19:51:40 chtekk Exp $
# The following options will be passed to all MySQL clients
[client]
port = 3306
[mysql]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8
[mysqladmin]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8
[mysqlcheck]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8
[mysqldump]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8
[mysqlimport]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8
[mysqlshow]
character-sets-dir=/usr/share/mysql/charsets
default-character-set=utf8
[myisamchk]
character-sets-dir=/usr/share/mysql/charsets
[myisampack]
character-sets-dir=/usr/share/mysql/charsets
[mysqld_safe]
err-log = /db/mysql/log/mysql.err
# To allow table cache to be raised
open-file-limit = 4096
[mysqld]
max_connections = 300
innodb_file_per_table = 1
log-slow-queries = /db/mysql/log/slow_query.log
long_query_time = 2000000
ft_min_word_len = 3
max_heap_table_size = 64M
tmp_table_size = 64M
server-id = 1
log-bin = /db/mysql/master-bin
log-bin-index = /db/mysql/master-bin.index
# END master/slave configuration
character-set-server = utf8
default-character-set = utf8
user = mysql
port = 3306
socket = /var/run/mysqld/mysqld.sock
pid-file = /var/run/mysqld/mysqld.pid
log-error = /db/mysql/log/mysqld.err
basedir = /usr
datadir = /db/mysql
key_buffer = 32M
max_allowed_packet = 32M
table_cache = 1024
thread_cache = 512
sort_buffer_size = 100M
net_buffer_length = 64K
read_buffer_size = 1M
read_rnd_buffer_size = 1M
myisam_sort_buffer_size = 100M
myisam_max_sort_file_size = 2G
myisam_repair_threads = 1
language = /usr/share/mysql/english
# security:
# using "localhost" in connects uses sockets by default
# skip-networking
# bind-address = 127.0.0.1
# point the following paths to different dedicated disks
tmpdir = /mnt/mysql/tmp
# log-update = /path-to-dedicated-directory/hostname
# you need the debug USE flag enabled to use the following directives,
# if needed, uncomment them, start the server and issue
# #tail -f /tmp/mysqld.sql /tmp/mysqld.trace
# this will show you *exactly* what's happening in your server ;)
#log = /tmp/mysqld.sql
#gdb
#debug = d:t:i:o,/tmp/mysqld.trace
#one-thread
# the rest of the innodb config follows:
# don't eat too much memory, we're trying to be safe on 64Mb boxes
# you might want to bump this up a bit on boxes with more RAM
innodb_buffer_pool_size = 1275M
# this is the default, increase it if you have lots of tables
innodb_additional_mem_pool_size = 16M
#
# i'd like to use /var/lib/mysql/innodb, but that is seen as a database :-(
# and upstream wants things to be under /var/lib/mysql/, so that's the route
# we have to take for the moment
#innodb_data_home_dir = /var/lib/mysql/
#innodb_log_arch_dir = /var/lib/mysql/
#innodb_log_group_home_dir = /var/lib/mysql/
# you may wish to change this size to be more suitable for your system
# the max is there to avoid run-away growth on your machine
innodb_data_file_path = ibdata1:20M:autoextend
# we keep this at around 25% of of innodb_buffer_pool_size
# sensible values range from 1MB to (1/innodb_log_files_in_group*innodb_buffer_pool_size)
innodb_log_file_size = 96M
# this is the default, increase it if you have very large transactions going on
innodb_log_buffer_size = 8M
# this is the default and won't hurt you
# you shouldn't need to tweak it
innodb_log_files_in_group = 2
# see the innodb config docs, the other options are not always safe
# This is not good for performance when used with bin_sync. Disabling.
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
innodb_lock_wait_timeout = 50
query_cache_size = 16M
query_cache_type = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
# uncomment the next directive if you are not familiar with SQL
#safe-updates
[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
ft_min_word_len = 3
[mysqlhotcopy]
interactive-timeout
答案 0 :(得分:2)
对于它的价值,11 megs free是完全没问题的。这是11兆的内存没有被用于任何东西,并且就硬件而言“浪费”了。实数是缓存中使用的“1380”,加上未使用的11兆。如有必要,可以吹走高速缓存。
您的系统有近1400 MB的RAM。
答案 1 :(得分:1)
您必须查看 myisam_max_sort_file_size 和 myisam_sort_buffer_size
的设置如果所有键的总和小于myisam_max_sort_file_size,那么在最坏的情况下,排序将在MyISAM表中着陆,这是一件好事。
否则,它将恢复为keycache。这意味着将必要的.MYI索引页加载到密钥缓存中并遍历内存中的索引页。没人想要!!!!
此变量的当前设置为2G。
查看正在构建的密钥。添加它们。如果所有密钥大小的总和超过2G,那么keycache一路!!!你将不得不提高这个价值。您可以使用
将会话值增加到4GSET myisam_max_sort_file_size = 1024 * 1024 * 1024 * 4;
SET myisam_sort_buffer_size = 1024 * 1024 * 1024 * 4;
或者您可以像这样直接种植数字:
SET myisam_max_sort_file_size = 4294967296;
SET myisam_sort_buffer_size = 4294967296;
在执行ENABLE KEYS之前;
如果您只是对备份数据感兴趣,为什么要将其开始索引?尝试使用 ARCHIVE 存储引擎。它没有任何索引。执行以下操作:
CREATE TABLE mytable_backup LIKE mytable;
ALTER TABLE mytable_backup ENGINE=ARCHIVE;
INSERT INTO mytable_backup SELECT * FROM mytable;
我还注意到您使用的是Amazon EC2。我之前从未去过EC2。运行此命令:
SHOW ENGINES;
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| CSV | YES | CSV storage engine | NO | NO | NO |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| FEDERATED | YES | Federated MySQL storage engine | NO | NO | NO |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| MyISAM | YES | Default engine as of MySQL 3.23 with great performance | NO | NO | NO |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
如果ARCHIVE存储引擎出现在列表中且支持为是,则可以选择备份到ARCHIVE表。如果没有,则必须调整myisam_max_sort_file_size和myisam_sort_buffer_size。
答案 2 :(得分:1)
你可以尝试
create table backup_table as (select * from production table) engine=myisam
这应该创建只包含数据而不包含任何键的表。然后,您可以通过执行
来添加密钥alter table backup_table add index(column_name)
我已经成功完成了几次,并且通常比使用钥匙插入速度快2倍。