我们为我们管理客户的网站,该网站正在处理严重的性能问题。该网站是在CMS内构建的,同时最多可处理20个访问者。只要网站的访问者是该网站的唯一用户,一切都很顺利。 CMS构建缓存,并将mysql查询的使用保持在最低限度。
当我们的客户的员工开始自己在网站上工作时,性能开始缺乏。整个工作流程都通过站点进行,包括更新数据库中的记录并插入新记录。当记录得到更新时,删除或插入的缓存会被删除,因此网站访问者的操作将被提交给查询。
我的第一个问题是mysql应该能够处理这些请求吗? 即使数据库的设计不好(见下文)。 当我在mysql中查看进程列表时,我看到简单的查询最多需要15秒。 当有更新或插入查询同时进行时,这些查询似乎只需要很长时间。
在持有所有记录的大表下面。 网站前端使用的所有内容都存储在这一个表中。 包括binarys。该表共有8,676条记录。 前端有很多使用WHERE的查询,例如Variabele5或Variabele6 这些Variabele字段的值取决于contenttype。
WHERE是例如:
WHERE Variabele5 > 500 AND Variabele6 =2 AND contenttype = 35
所有员工都在工作的环境不会被缓存。 在此环境中,有一些列表包含订单和搜索选项。 这些查询尽可能地进行memcached,但是当记录更新,删除或插入时,memcache将被清除。这与插入,更新和删除操作相结合可能是瓶颈。
所以我的问题是mysql应该能够处理这一切,还是数据库的设计不好处理请求。
+---------------+---------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +---------------+---------------+------+-----+---------+----------------+ | nr | mediumint(80) | NO | PRI | NULL | auto_increment | | hidden | int(1) | NO | | 0 | | | Title | varchar(255) | NO | MUL | | | | Body | text | NO | | NULL | | | Description | text | NO | | NULL | | | user_id | mediumint(80) | NO | | 0 | | | addate | varchar(255) | NO | | NULL | | | moddate | varchar(255) | NO | | NULL | | | contenttype | mediumint(80) | NO | MUL | 0 | | | parent | mediumint(80) | NO | | 0 | | | Status | mediumint(80) | NO | MUL | NULL | | | CODE | varchar(255) | NO | MUL | NULL | | | menu | mediumint(80) | NO | | NULL | | | filetype | mediumint(80) | NO | | NULL | | | folder | int(11) | NO | | NULL | | | sortIndex | bigint(255) | NO | | NULL | | | servercache | int(1) | NO | | 1 | | | futurecache | int(1) | NO | | NULL | | | publishFrom | date | YES | | NULL | | | publishUntil | date | YES | | NULL | | | pinned | int(11) | YES | | NULL | | | keywords | text | NO | | NULL | | | latestversion | int(11) | NO | | NULL | | | permalink | text | NO | | NULL | | | Integer1 | int(12) | NO | | 0 | | | Integer2 | int(4) | NO | | 0 | | | Integer3 | int(4) | NO | | 0 | | | Integer4 | int(4) | NO | | 0 | | | Variabele1 | varchar(255) | NO | | | | | Variabele2 | varchar(255) | NO | | | | | Variabele3 | varchar(255) | NO | | | | | Variabele4 | varchar(255) | NO | | | | | Date1 | varchar(255) | NO | | | | | Date2 | varchar(255) | NO | | | | | Text1 | text | NO | | NULL | | | Text2 | text | NO | | NULL | | | Text3 | text | NO | | NULL | | | Binary1 | longblob | NO | | NULL | | | Binary1Type | varchar(255) | NO | | NULL | | | Binary2 | longblob | NO | | NULL | | | Binary2Type | varchar(255) | NO | | NULL | | | Binary3 | longblob | NO | | NULL | | | Binary3Type | varchar(255) | NO | | NULL | | | browseAccess | varchar(255) | NO | | NULL | | | Binary4 | longblob | NO | | NULL | | | Binary4Type | varchar(255) | NO | | NULL | | | Binary5 | longblob | NO | | NULL | | | Binary5Type | varchar(255) | NO | | NULL | | | Binary6 | longblob | NO | | NULL | | | Binary6Type | varchar(255) | NO | | NULL | | | Integer5 | int(11) | NO | | NULL | | | Integer6 | int(11) | NO | | NULL | | | Variabele6 | varchar(255) | NO | | NULL | | | Binary7 | longblob | NO | | NULL | | | Binary7Type | varchar(255) | NO | | NULL | | | Binary8 | longblob | NO | | NULL | | | Binary8Type | varchar(255) | NO | | NULL | | | Binary9 | longblob | NO | | NULL | | | Binary9Type | varchar(255) | NO | | NULL | | | Binary10 | longblob | NO | | NULL | | | Binary10Type | varchar(255) | NO | | NULL | | | Binary11 | longblob | NO | | NULL | | | Binary11Type | varchar(255) | NO | | NULL | | | Binary12 | longblob | NO | | NULL | | | Binary12Type | varchar(255) | NO | | NULL | | | Binary13 | longblob | NO | | NULL | | | Binary13Type | varchar(255) | NO | | NULL | | | Binary14 | longblob | NO | | NULL | | | Binary14Type | varchar(255) | NO | | NULL | | | Binary15 | longblob | NO | | NULL | | | Binary15Type | varchar(255) | NO | | NULL | | | Text4 | text | NO | | NULL | | | Text5 | text | NO | | NULL | | | Text6 | text | NO | | NULL | | | Text7 | text | NO | | NULL | | | Text8 | text | NO | | NULL | | | Variabele5 | varchar(255) | NO | | NULL | | | Variabele7 | varchar(255) | NO | | NULL | | | Variabele8 | varchar(255) | NO | | NULL | | | Variabele9 | varchar(255) | NO | | NULL | | | Variabele10 | text | NO | | NULL | | | Variabele11 | varchar(255) | NO | | NULL | | | Variabele12 | varchar(255) | NO | | NULL | | | Variabele13 | varchar(255) | NO | | NULL | | | Variabele14 | varchar(255) | NO | | NULL | | | Variabele15 | varchar(255) | NO | | NULL | | | Variabele16 | varchar(255) | NO | | NULL | | | Variabele17 | varchar(255) | NO | | NULL | | | Variabele18 | varchar(255) | NO | | NULL | | | Variabele19 | varchar(255) | NO | | NULL | | | Variabele20 | varchar(255) | NO | | NULL | | | Variabele21 | varchar(255) | NO | | NULL | | | Variabele22 | varchar(255) | NO | | NULL | | | Variabele23 | varchar(255) | NO | | NULL | | | Variabele24 | varchar(255) | NO | | NULL | | | Variabele25 | varchar(255) | NO | | NULL | | | Variabele26 | varchar(255) | NO | | NULL | | | Variabele27 | varchar(255) | NO | | NULL | | | Variabele28 | varchar(255) | NO | | NULL | | | Variabele29 | varchar(255) | NO | | NULL | | | Variabele30 | varchar(255) | NO | | NULL | | | Variabele31 | varchar(255) | NO | | NULL | | | Variabele32 | varchar(255) | NO | | NULL | | | Variabele33 | varchar(255) | NO | | NULL | | | Variabele34 | varchar(255) | NO | | NULL | | | Variabele35 | varchar(255) | NO | | NULL | | | Variabele36 | varchar(255) | NO | | NULL | | | Variabele37 | varchar(255) | NO | | NULL | | | Variabele38 | varchar(255) | NO | | NULL | | | Variabele39 | varchar(255) | NO | | NULL | | | Variabele40 | varchar(255) | NO | | NULL | | | Variabele41 | varchar(255) | NO | | NULL | | | Variabele42 | varchar(255) | NO | | NULL | | | Variabele43 | varchar(255) | NO | | NULL | | | Variabele44 | varchar(255) | NO | | NULL | | | Variabele45 | varchar(255) | NO | | NULL | | | Variabele46 | varchar(255) | NO | | NULL | | | Variabele47 | varchar(255) | NO | | NULL | | | Variabele48 | varchar(255) | NO | | NULL | | | Variabele49 | varchar(255) | NO | | NULL | | | Variabele50 | varchar(255) | NO | | NULL | | +---------------+---------------+------+-----+---------+----------------+
索引:
+-------------+--------------+--------------+-------------+------+ | Key_name | Seq_in_index | Column_name | Cardinality | Null | +-------------+--------------+--------------+-------------+------+ | PRIMARY | 1 | nr | 8675 | | | Status | 1 | Status | 5 | | | Status | 2 | publishFrom | 8675 | YES | | Status | 3 | publishUntil | 8675 | YES | | CODE | 1 | CODE | 4337 | | | CODE | 2 | Title | 4337 | | | contenttype | 1 | contenttype | 30 | | | Title | 1 | Title | 2891 | | +-------------+--------------+--------------+-------------+------+
服务器信息: 4x英特尔(R)Xeon(R)CPU L5630 @ 2.13GHz
+---------------------------------------------------------------------------+ | free -mt | +---------------------------------------------------------------------------+ | total used free shared buffers cached | | Mem: 4096 3841 254 0 44 1223 | | -/+ buffers/cache: 2573 1522 | | Swap: 1023 422 601 | | Total: 5119 4263 856 | +---------------------------------------------------------------------------+
如果我需要提供任何其他信息,请告诉我。
编辑:一些示例查询
查询各不相同,但这里有一些例子:
搜索员工的查询:
EXPLAIN SELECT profiel.nr as nr, CONCAT(profiel.title,' ',profiel.Variabele49) as naam,profiel.Variabele3 as tel, profiel.Variabele44 as huurprijs, profiel.Variabele5 as inkomen, profiel.Variabele39 as personen, profiel.Variabele46 as perdatum, profiel.addate as inschrijving, profiel.text1 as opmerkingen, medewerker.Title as m_naam,profiel.Variabele48 as lang FROM site_content as profiel left join vw_activeContent as medewerker on medewerker.nr = profiel.Variabele9 WHERE profiel.contenttype =26 AND (profiel.Status=3 OR profiel.Text8='Nee') AND ( profiel.nr LIKE '%Rem%' OR profiel.title LIKE '%Rem%' OR ' ' LIKE '%Rem%' OR CONCAT(profiel.title,' ',profiel.Variabele49) LIKE '%Rem%' OR profiel.Variabele49 LIKE '%Rem%' OR profiel.Variabele3 LIKE '%Rem%' OR profiel.Variabele44 LIKE '%Rem%' OR profiel.Variabele5 LIKE '%Rem%' OR profiel.Variabele39 LIKE '%Rem%' OR profiel.Variabele46 LIKE '%Rem%' OR profiel.addate LIKE '%Rem%' OR profiel.text1 LIKE '%Rem%' OR medewerker.Title LIKE '%Rem%' OR profiel.Variabele48 LIKE '%Rem%' OR profiel.Variabele1 LIKE '%Rem%' OR profiel.Variabele3 LIKE '%Rem%' ) ORDER BY profiel.sortIndex
+----+-------------+-------+--------------+--------+----------------+--------------------+-------------+---+---------+--------------------+---+-------------+-----------------------------+-------+ | id | select_type | table | | type | possible_keys | | key | | key_len | ref | | | rows | Extra | +----+-------------+-------+--------------+--------+----------------+--------------------+-------------+---+---------+--------------------+---+-------------+-----------------------------+-------+ | 1 | SIMPLE | | profiel | | ref | Status,contenttype | contenttype | 3 | const | | | 1700 | Using where; Using filesort | | | 1 | SIMPLE | | site_content | eq_ref | PRIMARY,Status | | PRIMARY | | 3 | profiel.Variabele9 | 1 | Using where | | | +----+-------------+-------+--------------+--------+----------------+--------------------+-------------+---+---------+--------------------+---+-------------+-----------------------------+-------+
搜索查询访问者
EXPLAIN SELECT nr, title AS adres, Description AS description, Binary3 AS bin, Variabele2 AS
TYPE , Text3, Text2 AS verhuurd, Integer2 AS kamers, Integer3 AS personen, Variabele4 AS inclusief, Text1 AS oplevering, Integer5 AS huurpijs, Variabele6 AS wijk, moddate
FROM vw_activeContent
WHERE contenttype =22
AND Integer2 >=2
AND Integer3 >=1
AND Integer5 >380
AND Integer5 <770
ORDER BY Integer5 ASC
+----+-------------+-------+--------------+------+--------------------+-------------+-----+-------+---------+-----------------------------+------+-------+ | id | select_type | table | | type | possible_keys | | key | | key_len | ref | rows | Extra | +----+-------------+-------+--------------+------+--------------------+-------------+-----+-------+---------+-----------------------------+------+-------+ | 1 | SIMPLE | | site_content | ref | Status,contenttype | contenttype | 3 | const | 696 | Using where; Using filesort | | | +----+-------------+-------+--------------+------+--------------------+-------------+-----+-------+---------+-----------------------------+------+-------+
广泛使用的查询:
EXPLAIN SELECT DISTINCT Variabele2
FROM site_content
WHERE contenttype =22
AND STATUS =1
ORDER BY Variabele2 ASC
+----+-------------+-------+--------------+------+--------------------+-------------+-----+-------+---------+----------------------------------------------+------+-------+ | id | select_type | table | | type | possible_keys | | key | | key_len | ref | rows | Extra | +----+-------------+-------+--------------+------+--------------------+-------------+-----+-------+---------+----------------------------------------------+------+-------+ | 1 | SIMPLE | | site_content | ref | Status,contenttype | contenttype | 3 | const | 696 | Using where; Using temporary; Using filesort | | | +----+-------------+-------+--------------+------+--------------------+-------------+-----+-------+---------+----------------------------------------------+------+-------+
答案 0 :(得分:1)
从您给出的唯一查询示例中,Variabele5和Variabele6未编入索引,并且可能有大量contenttype = 35的记录,因此查询可能不会非常有效地使用索引。更多查询示例可能有助于我们获得更好的图片。
但是,表大小似乎不是很大,这让我觉得实际重建缓存可能是实际原因。因此,您可能希望查看应用程序中使用的缓存策略。我不知道你可以做多少测试,但我会尝试在关闭缓存的情况下测试你的场景。
答案 1 :(得分:0)
首先,正如你所说,你的数据库很乱。但是,鉴于您的硬件和相对较小的数据,我希望查询速度非常快。
如果查询速度很慢,可能是因为“where”条件的通配符 - 例如profiel.title LIKE '%Rem%'
。您可以使用全文搜索替换它。
另一个设计问题是在数据库中存储二进制文件;根据二进制文件的大小,这会对磁盘IO性能产生很大影响,并且 - 通常 - 二进制文件的编码/解码(通常需要数据库连接保持打开状态)可能比读取文本要慢得多/结果集中的数字。考虑在文件系统上缓存二进制文件,只有当知道它们已经更改时才会转到它们的数据库。
但是,在深入挖掘数据库之前,我认为您需要找到一种解决性能问题的方法,因此您知道正在解决正确的问题。考虑使用profiler来找到瓶颈,然后处理特定问题。