内存优化中的MYSQL只读DB

时间:2016-07-25 18:58:24

标签: mysql node.js database

我为生物化学研究人员创建了一个网站,允许用户查询特定基因并计算不同基因组之间的各种统计数据。

MySQL数据库是16GB且只读(我们的实验室生成了这些新颖的数据,网站是查看此数据的门户网站)。我对网站进行了性能测试,并意识到数据库查询是应用程序中最慢的部分。我想把整个数据库放到内存中,但是在普遍接受的解决方案中遇到了一些问题:

我有一个8核32GB服务器供使用。

选项1:设置ENGINE = MEMORY

有些列的类型为mediumtext,超出了64k行限制,拒绝加入MEMORY ENGINE

选项2:增加innodb_buffer_pool_size

这似乎并没有将数据实际存入内存。我使用所描述的技术(https://dba.stackexchange.com/questions/27328/how-large-should-be-mysql-innodb-buffer-pool-size)检查了使用过的缓冲池,但只看到了~100Mbs的使用缓冲池。 innodb_buffer_pool_size已正确设置为24GB。

选项3:创建RAMdisk并将数据库放在那里 基于少数SO帖子,这似乎不是一个很好的选择。

我该怎么办?请指教。

1 个答案:

答案 0 :(得分:1)

您是否检查了EXPLAIN和分析?

用于分析的样本

在这里,您可以看到服务器花费时间并优化此

的位置
MariaDB [yourschema]> set profiling=on;
Query OK, 0 rows affected (0.00 sec)

MariaDB [yourschema]> select * from table2 t1 left join table2 t2 ON t1.id <> t2.id;;
+--------+------------+--------+------------+
| id     | val        | id     | val        |
+--------+------------+--------+------------+
| 000002 | tabe 2 --2 | 000001 | tabe 2 --1 |
....
| 000005 | tabe 2 --5 | 000004 | tabe 2 --4 |
| 000006 | tabe 2 --6 | 000004 | tabe 2 --4 |
| 000001 | tabe 2 --1 | 000005 | tabe 2 --5 |
| 000002 | tabe 2 --2 | 000005 | tabe 2 --5 |
| 000004 | tabe 2 --4 | 000005 | tabe 2 --5 |
| 000006 | tabe 2 --6 | 000005 | tabe 2 --5 |
| 000001 | tabe 2 --1 | 000006 | tabe 2 --6 |
| 000002 | tabe 2 --2 | 000006 | tabe 2 --6 |
| 000004 | tabe 2 --4 | 000006 | tabe 2 --6 |
| 000005 | tabe 2 --5 | 000006 | tabe 2 --6 |
+--------+------------+--------+------------+
20 rows in set (0.00 sec)


MariaDB [yourschema]> show profile all;
+----------------------+----------+----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
| Status               | Duration | CPU_user | CPU_system | Context_voluntary | Context_involuntary | Block_ops_in | Block_ops_out | Messages_sent | Messages_received | Page_faults_major | Page_faults_minor | Swaps | Source_function       | Source_file   | Source_line |
+----------------------+----------+----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
| starting             | 0.000113 | 0.000072 |   0.000041 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 2 |     0 | NULL                  | NULL          |        NULL |
| checking permissions | 0.000006 | 0.000003 |   0.000003 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | check_access          | sql_parse.cc  |        6051 |
| checking permissions | 0.000005 | 0.000004 |   0.000001 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | check_access          | sql_parse.cc  |        6051 |
| Opening tables       | 0.000307 | 0.000071 |   0.000236 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 9 |     0 | open_tables           | sql_base.cc   |        4509 |
| After opening tables | 0.000010 | 0.000006 |   0.000003 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | open_tables           | sql_base.cc   |        4747 |
| System lock          | 0.000005 | 0.000004 |   0.000002 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_lock_tables     | lock.cc       |         308 |
| Table lock           | 0.000010 | 0.000009 |   0.000001 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_lock_tables     | lock.cc       |         313 |
| init                 | 0.000027 | 0.000025 |   0.000002 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_select          | sql_select.cc |        3427 |
| optimizing           | 0.000014 | 0.000012 |   0.000001 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | optimize_inner        | sql_select.cc |        1092 |
| statistics           | 0.000022 | 0.000021 |   0.000002 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | optimize_inner        | sql_select.cc |        1373 |
| preparing            | 0.000117 | 0.000037 |   0.000084 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 4 |     0 | optimize_inner        | sql_select.cc |        1398 |
| executing            | 0.000010 | 0.000004 |   0.000003 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | exec_inner            | sql_select.cc |        2551 |
| Sending data         | 0.000106 | 0.000102 |   0.000003 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | exec_inner            | sql_select.cc |        3223 |
| end                  | 0.000007 | 0.000004 |   0.000003 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_select          | sql_select.cc |        3462 |
| query end            | 0.000008 | 0.000006 |   0.000001 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_execute_command | sql_parse.cc  |        5687 |
| closing tables       | 0.000004 | 0.000003 |   0.000002 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | close_thread_tables   | sql_base.cc   |         935 |
| Unlocking tables     | 0.000008 | 0.000007 |   0.000001 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_unlock_tables   | lock.cc       |         395 |
| freeing items        | 0.000008 | 0.000006 |   0.000002 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | mysql_parse           | sql_parse.cc  |        7319 |
| updating status      | 0.000023 | 0.000008 |   0.000015 |                 0 |                   0 |            0 |             0 |             1 |                 0 |                 0 |                 0 |     0 | dispatch_command      | sql_parse.cc  |        1937 |
| cleaning up          | 0.000004 | 0.000002 |   0.000002 |                 0 |                   0 |            0 |             0 |             0 |                 0 |                 0 |                 0 |     0 | dispatch_command      | sql_parse.cc  |        1956 |
+----------------------+----------+----------+------------+-------------------+---------------------+--------------+---------------+---------------+-------------------+-------------------+-------------------+-------+-----------------------+---------------+-------------+
20 rows in set (0.00 sec)

MariaDB [yourschema]>