PHP PDO巨大的数据库查询速度优化

时间:2013-01-12 06:51:45

标签: php mysql pdo mysqli

我正在建立一个网站geekwiz.com。测试加载它,LONG。

我是php的新手,只是从mysql_query更改为PDO。我必须做多个查询才能在第一页上显示产品,并且该表有超过150万条记录。

如何更快地完成这项工作?

以下是我创建的代码:

#connection
$DBH = new PDO("mysql:host=$host;dbname=$dbname", $user, $pass);
$DBH->setAttribute(PDO::ATTR_EMULATE_PREPARES, 0);

单独的文件

#fetch data
require_once('db_config.php');
$STH = $DBH->query("
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('notebook pc - other') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Desktop PC') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Hard Drives - External') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Camcorders - Analog/Digital') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Monitors - LCD Flat Panel') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Software - PC Games') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('third party accessories') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE advertisercategory LIKE ('LED TV') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Motherboards - %') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE advertisercategory LIKE ('LED TV') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Hard Drives - External') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('CPU Cooling') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('CPU Thermal Paste / Grease') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Keyboards') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Memory (USB Flash Drive)') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Memory (Desktop Memory)') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Memory (Notebook Memory)') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Mouse') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Printer - Inkjet Printers') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Projectors') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Processors - Desktops') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9;

");

$STH->setFetchMode(PDO::FETCH_OBJ);

while ($row = $STH->fetch()) {
echo "<ul>";
echo "<li>" .$row->name ."</li>";
echo "<li>" .$row->buyurl ."</li>";
echo "<li>" .$row->imageurl ."</li>";
echo "<li>" .$row->price ."</li>";
echo "<li>" .$row->manufacturer ."</li>";
echo "<li>" .$row->manufacturerid ."</li>";
echo "</ul>";
}

3 个答案:

答案 0 :(得分:2)

一些问题,你可能不喜欢我要说的。

按文本/ varchar排序是很慢的,并且更新将需要永远。你应该做的是将所有文本转换为整数。因此advertisercategory不应包含实际文本。相反,它应该是固定长度INT,它链接到存储实际文本的表。 specialinstock也是如此。 standardshippingcost看起来像一个varchar,因为它是单引号,如果它看起来像那样,那真的不应该是这种情况。如果它是数字,它应该是数字。我对您的架构不太了解,因此standardshippingcost可能需要位于由FK链接到catalog的“自己的表格中”。在{unix time'中lastupdated应为INT

你的数据应该是INT的原因是因为1)它比notebook pc - other占用更少的空间(因此排序时的内存更少)和2)更新会更快(我假设你'重新使用InnoDB)。

因此,当您排序SELECT所有id(应该是自动增量PRIMARY),advertisercategoryinstock时, standardshippingcoststandardshippingcostlastupdated到PHP数组中。找到最有效的方式来获得你想要的东西(你会发现快捷方式)。最后,通过SELECT密钥PRIMARY SELECT特定数据。

我会倒退解释。这种方法产生最高性能的原因是因为如果你在InnoDB表上PRIMARY没有任何其他索引advertisercategory,那么表的大小无关紧要,它将返回“立即”。正如您所经历的那样,使用巨大而深奥的查询很慢。这就是你应该如何实际检索数据。

PHP比逻辑上的MySQL更快。这就是它的用途。堆栈上有很多Q显示这个。这就是你想要对PHP进行排序的原因。另外,如果您在数据中看到某种统计或其他类型的模式,您实际上可以编写更好的算法来使您的排序运行得更快。

除了unix日期之外,您的其他列不应该太长。 notebook pc - other可能会变得有点大,但与19已经长度为advertisercategory相比,它不会有任何变化。我在您的Q中看到大约30个INT,所以这只是LENGTH 2 UPDATE,因为这是100个潜在值。这将在排序时节省空间和内存。

此外,在更新时使用固定长度的int会更快,因为群集表将始终保持紧密和碎片整理。如果您在'Memory (USB Flash Drive)''Projectors'的群集InnoDB中间1行,您只是让数据库整理整个数据库以消除您刚刚创建的差距。如果你在一个固定长度的int列中从一个10更改为INT,那么你就不会这样做,因为两者都占用相同的空间,没有间隙,不需要群集的defrags。

当你这样做时,你会惊讶于大量的性能提升,但是,这将涉及更多的代码。那你的优先级,表现或开发时间是什么?


呐喊!忘了unix时间解释。您希望使用此功能,因为您可以将其存储在DECIMAL中持续数秒,将{{1}}存储为微秒。这样做可以在PHP中进行最简单的操作,因为php在unix中。

答案 1 :(得分:0)

嗯,首先,PDO与巨大的数据库速度优化完全无关 为了让它更清晰一点,想象一下你问的问题类似于“如何驾驶我的福特护航从纽约布鲁克林到佛罗里达州坦帕”。汽车制造在这里无关紧要。 DB驱动程序也是如此。

现在到你的数据库。
你有很多工作要做。

首先,绝对没办法使用Mysql FULLTEXT制作类别。它应该只是一个数字索引,链接到具有类别结构的表。

因此,查询变得像这样

SELECT * FROM catalog WHERE category=1 AND special = 'YES' AND instock = 'YES' 
AND standardshippingcost=0 ORDER BY lastupdated DESC limit 3;

并且,如果正确索引,它可以非常快。比如说,如果特别优惠的数量明显受限,无论数据库中有多少条记录,此字段的索引都会使其快速生成。

但是,整个数据库架构可能会有所不同。除了可以采用其他方式进行优化之外。但我们对您的项目知之甚少并不确定,老实说,这不是简单的Q&amp; A格式的问题。

答案 2 :(得分:0)

编写一个名为saveCacheData.php的php脚本来执行该组sql查询,该脚本还会将数据转储到名为cacheData.php的文件中。

在主页面上,将sql查询替换为require_once('cacheData.php')

现在,当更新mysql表catalog中的数据时,请确保您还执行file_get_contents('saveCacheData.php')(或执行某种将为您缓存数据的cron作业)。

顺便说一下,我认为你的数据库设计可以更好,但是将数据库数据缓存到磁盘通常也是一个好主意。