我为生物化学研究人员创建了一个网站,允许用户查询特定基因并计算不同基因组之间的各种统计数据。
MySQL数据库是16GB且只读(我们的实验室生成了这些新颖的数据,网站是查看此数据的门户网站)。我对网站进行了性能测试,并意识到数据库查询是应用程序中最慢的部分。我想把整个数据库放到内存中,但是在普遍接受的解决方案中遇到了一些问题:
我有一个8核32GB服务器供使用。
选项1:设置ENGINE = MEMORY
有些列的类型为mediumtext
,超出了64k行限制,拒绝加入MEMORY ENGINE
选项2:增加innodb_buffer_pool_size
这似乎并没有将数据实际存入内存。我使用所描述的技术(https://dba.stackexchange.com/questions/27328/how-large-should-be-mysql-innodb-buffer-pool-size)检查了使用过的缓冲池,但只看到了~100Mbs的使用缓冲池。 innodb_buffer_pool_size
已正确设置为24GB。
选项3:创建RAMdisk并将数据库放在那里 基于少数SO帖子,这似乎不是一个很好的选择。
我该怎么办?请指教。
答案 0 :(得分:1)
您是否检查了EXPLAIN和分析?
用于分析的样本
在这里,您可以看到服务器花费时间并优化此
的位置MariaDB [yourschema]> set profiling=on;
Query OK, 0 rows affected (0.00 sec)
MariaDB [yourschema]> select * from table2 t1 left join table2 t2 ON t1.id <> t2.id;;
+--------+------------+--------+------------+
| id | val | id | val |
+--------+------------+--------+------------+
| 000002 | tabe 2 --2 | 000001 | tabe 2 --1 |
....
| 000005 | tabe 2 --5 | 000004 | tabe 2 --4 |
| 000006 | tabe 2 --6 | 000004 | tabe 2 --4 |
| 000001 | tabe 2 --1 | 000005 | tabe 2 --5 |
| 000002 | tabe 2 --2 | 000005 | tabe 2 --5 |
| 000004 | tabe 2 --4 | 000005 | tabe 2 --5 |
| 000006 | tabe 2 --6 | 000005 | tabe 2 --5 |
| 000001 | tabe 2 --1 | 000006 | tabe 2 --6 |
| 000002 | tabe 2 --2 | 000006 | tabe 2 --6 |
| 000004 | tabe 2 --4 | 000006 | tabe 2 --6 |
| 000005 | tabe 2 --5 | 000006 | tabe 2 --6 |
+--------+------------+--------+------------+
20 rows in set (0.00 sec)
MariaDB [yourschema]> show profile all;
+----------------------+----------+----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
| Status | Duration | CPU_user | CPU_system | Context_voluntary | Context_involuntary | Block_ops_in | Block_ops_out | Messages_sent | Messages_received | Page_faults_major | Page_faults_minor | Swaps | Source_function | Source_file | Source_line |
+----------------------+----------+----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
| starting | 0.000113 | 0.000072 | 0.000041 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | NULL | NULL | NULL |
| checking permissions | 0.000006 | 0.000003 | 0.000003 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | check_access | sql_parse.cc | 6051 |
| checking permissions | 0.000005 | 0.000004 | 0.000001 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | check_access | sql_parse.cc | 6051 |
| Opening tables | 0.000307 | 0.000071 | 0.000236 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 9 | 0 | open_tables | sql_base.cc | 4509 |
| After opening tables | 0.000010 | 0.000006 | 0.000003 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | open_tables | sql_base.cc | 4747 |
| System lock | 0.000005 | 0.000004 | 0.000002 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_lock_tables | lock.cc | 308 |
| Table lock | 0.000010 | 0.000009 | 0.000001 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_lock_tables | lock.cc | 313 |
| init | 0.000027 | 0.000025 | 0.000002 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_select | sql_select.cc | 3427 |
| optimizing | 0.000014 | 0.000012 | 0.000001 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | optimize_inner | sql_select.cc | 1092 |
| statistics | 0.000022 | 0.000021 | 0.000002 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | optimize_inner | sql_select.cc | 1373 |
| preparing | 0.000117 | 0.000037 | 0.000084 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 0 | optimize_inner | sql_select.cc | 1398 |
| executing | 0.000010 | 0.000004 | 0.000003 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | exec_inner | sql_select.cc | 2551 |
| Sending data | 0.000106 | 0.000102 | 0.000003 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | exec_inner | sql_select.cc | 3223 |
| end | 0.000007 | 0.000004 | 0.000003 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_select | sql_select.cc | 3462 |
| query end | 0.000008 | 0.000006 | 0.000001 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_execute_command | sql_parse.cc | 5687 |
| closing tables | 0.000004 | 0.000003 | 0.000002 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | close_thread_tables | sql_base.cc | 935 |
| Unlocking tables | 0.000008 | 0.000007 | 0.000001 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_unlock_tables | lock.cc | 395 |
| freeing items | 0.000008 | 0.000006 | 0.000002 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | mysql_parse | sql_parse.cc | 7319 |
| updating status | 0.000023 | 0.000008 | 0.000015 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | dispatch_command | sql_parse.cc | 1937 |
| cleaning up | 0.000004 | 0.000002 | 0.000002 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | dispatch_command | sql_parse.cc | 1956 |
+----------------------+----------+----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
20 rows in set (0.00 sec)
MariaDB [yourschema]>