从现在开始的几个月以来，我一直在从公开来源收集足球比赛的数据。为此，我使用PHP通过XPath获取特定网站URL的数据，该数据显示了特定足球比赛的数据。然后我进行一些数据编辑，使它们适合我的需要。下一步是将它们转移到几张表中的MySQL数据库中。
随着数据库的快速增长，我慢慢遇到了严重的性能问题。因为我在计算机上本地进行所有操作，而这并不是机器上的庞然大物，所以处理比赛已经花费了一些时间。感觉有多快：在数据挖掘的第一天，一场比赛大约花了24秒。但是，现在的平均值已超过60秒的阈值。
到目前为止，我偶尔会进入PHP代码并尝试在可能的情况下对其进行改进，因为我认为主要的问题在于代码段不是那么干净。虽然有所帮助，但几天后的平均时间又增加了，最近我开始意识到肯定还有另一个耗时的问题。因此，我制作了一个测试PHP脚本，该脚本在运行主要代码时执行某种日志记录。
这表明我为将数据插入数据库表中而进行的一些SQL查询平均花费大量时间（我在这里分析了100个匹配项）：
再次检查查询，我意识到它们非常复杂。
涉及的表是
tblStartingSquad
+----+---------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+--------+
| id | matchID | player1ID | player2ID | player3ID | player4ID | player5ID | player6ID | player7ID | player8ID | player9ID | player10ID | player11ID | clubID |
+----+---------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+--------+
| 1 | 1 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 1 |
| 2 | 1 | 12 | 13 | 14 | 15 | 16 | 17 | 16 | 17 | 18 | 19 | 20 | 2 |
| 3 | 2 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 1 |
| 4 | 2 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 3 |
+----+---------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+--------+
tbl替代品
+----+---------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+--------+
| id | matchID | player12ID | player13ID | player14ID | player15ID | player16ID | player17ID | player18ID | player19ID | player20ID | player21ID | player22ID | player23ID | clubID |
+----+---------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+--------+
| 1 | 1 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 1 |
| 2 | 1 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 2 |
| 3 | 2 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 1 |
| 4 | 2 | 56 | 57 | 58 | 59 | 60 | 61 | 61 | 62 | 63 | 64 | 65 | 66 | 3 |
+----+---------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+--------+
tblMatch
+---------+---------------------+-------------+------------------+
| matchID | date | coach1 | coach2 |
+---------+---------------------+-------------+------------------+
| 1 | 2006-08-19 22:00:00 | Piotr Nowak | Fernando Clavijo |
| 2 | 2006-08-15 21:00:00 | Piotr Nowak | Mustafa Ugur |
+---------+---------------------+-------------+------------------+
tblPlayer
+----------+------------------------+------------------+
| playerID | namePlayer | short |
+----------+------------------------+------------------+
| 1 | Enis Ulusan | enis-ulusan |
| 2 | Grant Robert Murray | grant-murray |
| 3 | Evgeniy Shpedt | evgeniy-shpedt |
| 4 | Mihai Alexandru Costea | mihai-costea |
| 5 | Jan Zolna | jan-zolna |
| 6 | Adrian Gheorghiu | adrian-gheorghiu |
| 7 | Marius Marian Croitoru | marius-croitoru |
| 8 | Jacov Nachtailer | jacov-nachtailer |
| ... | ... | ... |
+----------+------------------------+------------------+
tblClub
+--------+-----------------+
| clubID | nameClub |
+--------+-----------------+
| 1 | D.C. United |
| 2 | Colorado Rapids |
| 3 | Caykur Rizespor |
+--------+-----------------+
这些是涉及的查询：
SQL查询开始小组
$tblstarting_squad = 'INSERT INTO tblStartingSquad (matchID, player1ID, player2ID, player3ID, player4ID, player5ID, player6ID, player7ID, player8ID, player9ID, player10ID, player11ID, clubID)
SELECT
(SELECT matchID FROM tblMatch WHERE date = "' . $match_date . '" AND coach1 = "' . $match_coach_home . '" AND coach2 = "' . $match_coach_away . '"),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[0] . '" AND short = "' . $player_short[0] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[1] . '" AND short = "' . $player_short[1] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[2] . '" AND short = "' . $player_short[2] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[3] . '" AND short = "' . $player_short[3] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[4] . '" AND short = "' . $player_short[4] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[5] . '" AND short = "' . $player_short[5] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[6] . '" AND short = "' . $player_short[6] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[7] . '" AND short = "' . $player_short[7] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[8] . '" AND short = "' . $player_short[8] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[9] . '" AND short = "' . $player_short[9] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[10] . '" AND short = "' . $player_short[10] . '" LIMIT 1),
(SELECT clubID FROM tblClub WHERE nameClub = "' . $match_club[1] . '" LIMIT 1)
WHERE NOT EXISTS (
SELECT e.matchID
FROM tblStartingSquad As e
INNER JOIN tblMatch As m
ON e.matchID = m.matchID
WHERE m.date = "' . $match_date . '" AND m.coach1 = "' . $match_coach_home . '" AND m.coach2 = "' . $match_coach_away . '" AND e.clubID = (SELECT clubID FROM tblClub WHERE nameClub = "' . $match_club[1] . '")
);';
if (!mysqli_query($db_connection, $tblstarting_squad)) {
echo("Error description $tblstarting_squad: " . mysqli_error($db_connection) . "<br />");
}
SQL查询基准替代品
$tblsubstitutes = 'INSERT INTO tblSubstitutes (matchID, player12ID, player13ID, player14ID, player15ID, player16ID, player17ID, player18ID, player19ID, player20ID, player21ID, player22ID, player23ID, clubID)
SELECT
(SELECT matchID FROM tblMatch WHERE date = "' . $match_date . '" AND coach1 = "' . $match_coach_home . '" AND coach2 = "' . $match_coach_away . '"),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[11] . '" AND short = "' . $player_short[11] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[12] . '" AND short = "' . $player_short[12] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[13] . '" AND short = "' . $player_short[13] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[14] . '" AND short = "' . $player_short[14] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[15] . '" AND short = "' . $player_short[15] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[16] . '" AND short = "' . $player_short[16] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[17] . '" AND short = "' . $player_short[17] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[18] . '" AND short = "' . $player_short[18] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[19] . '" AND short = "' . $player_short[19] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[20] . '" AND short = "' . $player_short[20] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[21] . '" AND short = "' . $player_short[21] . '" LIMIT 1),
(SELECT playerID FROM tblPlayer WHERE namePlayer = "' . $player_name[22] . '" AND short = "' . $player_short[22] . '" LIMIT 1),
(SELECT clubID FROM tblClub WHERE nameClub = "' . $match_club[1] . '" LIMIT 1)
WHERE NOT EXISTS (
SELECT e.matchID
FROM tblSubstitutes As e
INNER JOIN tblMatch As m
ON e.matchID = m.matchID
WHERE m.date = "' . $match_date . '" AND m.coach1 = "' . $match_coach_home . '" AND m.coach2 = "' . $match_coach_away . '" AND e.clubID = (SELECT clubID FROM tblClub WHERE nameClub = "' . $match_club[1] . '")
);';
if (!mysqli_query($db_connection, $tblsubstitutes)) {
echo("Error description $tblsubstitutes: " . mysqli_error($db_connection) . "<br />");
}
这两个查询实际上是相同的。如果没有其他具有相同数据的条目，他们会将11个（分别为12个）玩家的playerID插入tblStartingSquad（分别为tblSubstitutes）中。 playerID必须事先在数据库中检查，因为原始数据没有单独的ID。通过namePlayer表中的short和tblPlayer选择它即可。
表tblStartingSquad和tblSubstitutes本身当前包含110,000行（用于55,000个匹配项），tblPlayer位于100,000行。
我已经在Google上搜索了一些解决方案，但找不到能够提高整体速度的任何东西。我所理解的一个问题是，我必须单独检查每个玩家，因此我得到11和12个子查询。那不是很优雅，但是我真的不知道如何改善它。也许有人在StackOverflow上有建议吗？
答案 0 :(得分：3)
重新考虑您的宽表设计，以形成长表设计。带数字后缀的列永远不是理想的数据存储。行很便宜。柱子很贵。长格式的连接，聚合，搜索，索引等操作要容易得多。否则，当您显示12个子查询甚至自连接时，查询将变得很复杂！
有趣的是，您的 tblClub 和 tblPlayer 是长格式，但不是 tblStartingSquad 和 tblSubstitutes ！只需将所有无关的玩家列删除为一个，其中各行表示不同的玩家：
tblStartingSquad
ID MatchID PlayerID ClubID
1 1 5 1
2 1 8 1
3 1 9 1
...
tbl替代品
ID MatchID PlayerID ClubID
1 1 2 1
2 1 16 1
3 1 7 1
...
tblMatch （为清楚起见，已重命名教练列）
ID Date HomeCoach AwayCoach
1 2006-08-19 22:00:00 Piotr Nowak Fernando Clavijo
2 2006-08-15 21:00:00 Piotr Nowak Mustafa Ugur
PHP
通过这种数据库设计，您可以运行一个更简单的PHP参数化查询调用，并且甚至可以更方便地使用PDO（而不是mysqli）来绑定数组中的许多参数。
// OPEN CONNECTION
$dbconn = new PDO("mysql:host=$servername;dbname=$dbname", $username, $password);
// SET PDO ERROR MODE TO EXCEPTION
$dbconn -> setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
// PREPARED STATEMENT
$sql = "INSERT INTO tblStartingSquad (`Match`, `PlayerID`, `ClubID`)
SELECT m.MatchID, p1.PlayerID, c.ClubID
FROM
(SELECT p.PlayerID
FROM tblPlayer p
WHERE p.namePlayer IN (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
) p1
INNER JOIN
(SELECT p.PlayerID
FROM tblPlayer p
WHERE p.short IN (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
) p2 ON p1.PlayerID = p2.PlayerID
CROSS JOIN
(SELECT MatchID
FROM tblMatch
WHERE `date` = ? AND HomeCoach = ? AND AwayCoach = ?) m
CROSS JOIN
(SELECT ClubID
FROM tblClub
WHERE nameClub = ?) c
WHERE NOT EXISTS
(SELECT 1 FROM tblStartingSquad As e
WHERE e.MatchID = m.matchID)"
try {
// INITIALIZE STATEMENT
$stmt = $dbconn->prepare($sql);
$params = array($player_name[0], $player_name[1], $player_name[2],
$player_name[3], $player_name[4], $player_name[5],
$player_name[6], $player_name[7], $player_name[8],
$player_name[9], $player_name[10],
$player_short[0], $player_short[1], $player_short[2],
$player_short[3], $player_short[4], $player_short[5],
$player_short[6], $player_short[7], $player_short[8],
$player_short[9], $player_short[10],
$match_date, $match_coach_home, $match_coach_away, $match_club);
// ITERATIVELY BIND PARAMS
foreach($params as $key => $val) {
$stmt->bindParam($key+1, $val, PDO::PARAM_STR);
}
// EXECUTE ACTION
$stmt->execute();
} catch (PDOException $e) {
echo "Error: " . $e->getMessage();
}
对 tblSubstitutes 进行类似的调用，调整附加查询的目标以及WHERE子句和参数值。
答案 1 :(得分：1)
正如O.Jones在评论中已经指出的那样，查看更多（php）代码以判断性能问题将很有帮助。 尽管重新设计了数据库，但另一个快速建议实际上是运行一个循环，并为玩家ID的各个查询使用prepared statement。这样可能会稍微提高性能。
在我看来，在PHP中做更多的事情而不是将数据获取逻辑外包给SQL似乎是合乎逻辑的。
答案 2 :(得分：0)
语句并不复杂。它们只是包含很多查找。因此，请确保查找速度快。您需要以下索引。如果尚未将它们添加到数据库中。
create index idx_find_player on tblplayer (nameplayer, short, playerid);
create index idx_find_club on tblclub (nameclub, clubid)
create index idx_find_match on tblmatch (date, coach1, coach2, matchid)
create index idx_find_squad1 on tblstartingsquad (matchid, clubid)
create index idx_find_squad2 on tblSsartingsquad (clubid, matchid)
create index idx_find_subs1 on tblsubstitutes (matchid, clubid)
create index idx_find_subs2 on tblsubstitutes (clubid, matchid)
我不确定哪个小队索引更可能被使用，因此创建两个小队索引并查看哪个DBMS选择。然后，您可以放下另一个。替代索引也一样。
答案 3 :(得分：0)
要获得准确的答案，我们必须在SQL中查看您的执行计划，将其发送至此处，以便我为您解决问题
在此之前，我认为您做错了所有事情，而不是为单行写一个选择，您只需在数据库中定义用户定义表并传递值即可。通过这样做，您可能会在SQL和服务器端代码中获得更好的性能，并且我保证您在第一层上具有最佳性能，在那之后，正如我在执行计划之前所说的那样，
创建类型[dbo]。[com_ListOfGuid]如表（NamePlayer NVARCHAR（256）非空，短NVARCHAR（256）非空）
与其在代码中创建select语句，不如声明一个数据表并将其填充到数据中并像其他参数一样传递给它（不要忘记像您定义的表一样为您设置一个列名）类型） 3.在您的SQL代码中，只需将用户定义的表类型与表连接，然后将它们插入目标表中（我举一个例子，您可以根据需要进行更改）
创建过程Procedure_Your_Name @ObjectTable Type_Your_Name READONLY 如 插入TARGET_Table（） 选择 * 从 first_Table F INNER JOIN @ObjectTable打开F.NamePlayer = t.NamePlayer和t.Short = f.Short