我正在尝试向服务器查询大量数据,这是我的代码:
$queryString = "";
$connect = mysqli_connect("localhost", "username", "password", "database");
$loopLength = 20;
$currentGroup = 1;
$currentLoopAmount = 0;
$queryAmount = 5;
for($v = 0; $v < ceil($loopLength / $queryAmount); $v++){
//echo "Looping Main: " . $v . "<br>";
//echo $loopLength - (($currentGroup - 1) * 10) . "<br>";
if($loopLength - (($currentGroup - 1) * $queryAmount) >= $queryAmount){
$currentLoopAmount = $queryAmount;
}
else{
$currentLoopAmount = $loopLength - (($currentGroup - 1) * $queryAmount);
}
//echo $currentLoopAmount;
$queryString = "";
for($q = (($currentGroup - 1) * $queryAmount); $q < $currentLoopAmount + (($currentGroup - 1) * $queryAmount); $q++){
//echo " Looping Sub: " . $q . "<br>";
$tempVariable = grabPageData($URLs[$q], $q);
$queryString .= $tempVariable;
if($q < $loopLength-1){
$queryString .= ",";
}
else{
$queryString .= ";";
}
}
echo $queryString;
$query = "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES " . $queryString;
$result = mysqli_query($connect, $query);
if($result){
echo "Success";
}
else{
echo "Failed : " . mysqli_error($connect) . "<br>";
}
$currentGroup += 1;
}
$loopLength
变量是动态的,可以是数千或可能是数十万。我设计了这个功能,将大量的数字划分为一批较小的查询,因为我无法通过GoDaddy在我的共享主机服务上一次上传所有数据。 $queryAmount
变量表示较小查询的大小。
以下是插入表中的一个值集的示例:
来自公告的数据表明我的代码是在grabPageData()
函数中检索的。
('http://www.publicnoticeads.com/az/search/view.asp?T=PN&id=37/7292017_24266919.htm','Pima','Green Valley News and Sun','2017/07/30',' ___________________________ARIZONA SUPERIOR COURT, PIMA COUNTYIn the Matter of the Estate of:JOSEPH T, DILLARD, SR.,Deceased.DOB: 10/09/1931No. PB20170865NOTICE TO CREDITORS(FOR PUBLICATION)NOTICE IS HEREBY GIVEN that DANA ANN DILLARD CALL has been appointed Personal Representative of this Estate. All persons having claims against the Estate are required to present their claimswithin four months after the date of the firat publication of this notice or the claims will be forever barred. Claims must be presented by delivering or mailing a written statement of the claim to the Personal Representative at the Law Offices of Michael W. Murray, 257 North Stone Avenue, Tucson, Arizona 85701.DATED this 17th day of July, 2017./S/ Micahel W. MurrayAttorney for the Personal RepresentativePub: Green Valley News & SunDate: July 23, 30, August 6, 2017 Public Notice ID: 24266919',' 24266919'),
为了获得这些数据,我通过一个抓取页面并抓取它的函数来运行它。然后我把网页的html代码放到这个函数中:
function cleanData($data){
$data = strip_tags($data);
//$data = preg_replace("/[^a-zA-Z0-9]/", "", $data);
//$data = mysql_real_escape_string($data);
return $data;
}
如上所示,这给了我没有标签的内容。这是问题所在。
该功能执行,一切似乎都很花哨。那么函数(取决于$queryAmount
变量,出于问题的缘故我不会超过10)输出,你可以看到它会在函数中出现类似...
Failed : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1
奇怪的是,当我有大量数据时,例如$loopLength
变量就像116.结果将输出,“失败:(错误)失败:(错误)Fai ...(错误)成功。所以它实际上只查询最后一组数据???不确定。
我不知道如何解决这个问题,我想要一个清醒的眼睛。请有人帮帮我。我一直在研究这个问题......几个小时试图找到解决方案。
很抱歉让这个问题变得很痛苦:(
编辑:
我将代码从以前更改为使用mysql编写的语句,而不是......请参阅下文:
$grabDataResults = [
"url" => "",
"county" => "",
"paperco" => "",
"date" => "",
"notice" => "",
"id" => "",
];
$connect = mysqli_connect("localhost", "bwt_admin", "Thebeast1398", "NewCoDatabase");
if($stmt = mysqli_prepare($connect, "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES (?, ?, ?, ?, ?, ?)")){
mysqli_stmt_bind_param($stmt, 'ssssss', $grabDataResults["url"], $grabDataResults["county"], $grabDataResults["paperco"], $grabDataResults["date"], $grabDataResults["notice"], $grabDataResults["id"]);
$loopLength = 1;
for($v = 0; $v < $loopLength; $v++){
$grabDataResults = grabPageData($URLs[$v], $v);
mysqli_stmt_execute($stmt);
printf("%d Row inserted.\n", mysqli_stmt_affected_rows($stmt));
printf("Error:\n", mysqli_stmt_error($stmt));
echo "(" . $grabDataResults["url"] . "," . $grabDataResults["county"] . "," . $grabDataResults["paperco"] . "," . $grabDataResults["date"] . "," . $grabDataResults["notice"] . "," . $grabDataResults["id"] . ")";
}
mysqli_stmt_close($stmt);
mysqli_close($connect);
}
不幸的是,这是我从输出中获得的:
1 Row inserted. 0 Error:
实际上没有错误打印出来并且插入了行。但是,当我导航到我的数据库,并查看已存储的值时..它们都是空的。 echo语句输出:
(http://www.publicnoticeads.com/az/search/view.asp?T=PN&id=31/7292017_24266963.htm,Yuma,Sun (Yuma), The,2017/07/30,, 24266963)
所以我知道所有变量都包含除$notice
变量之外的其他变量,由于某种原因它被我的cleanData()
函数破坏。
答案 0 :(得分:0)
我可以在您的查询中看到的主要错误是查询本身。您正在使用带有分隔字段和值的INSERT INTO。但你忘了在价值观上使用pharentesis。
请记住,INSERT INTO的使用如下:
First option:
INSERT INTO table field1 = value1, field2 = value2;
Second option:
INSERT INTO table (field1, field2) VALUES (value1, value2);
另外,请记住转义每个字段和值以避免更多错误:示例:
First option:
INSERT INTO `table` `field1` = 'value1', `field2` = 'value2';
Second option:
INSERT INTO `table` (`field1`, `field2`) VALUES ('value1', 'value2');
如果您正在使用mysqli驱动程序,为了更加安全,您可以使用预准备语句来自动转义您的值。在这种情况下,查询的语法如下:
First option:
INSERT INTO `table` `field1` = ?, `field2` = ?;
Second option:
INSERT INTO `table` (`field1`, `field2`) VALUES (?, ?);
此外,您需要使用mysqli_prepare(),mysqli_bind_param()和mysqli_execute(),而不是使用mysqli_query()。您可以在此处查看有关其语法的更多数据:http://php.net/manual/en/mysqli.prepare.php
至少,您可以使用mysqli_real_escape_string()函数来正确转义和检查您的输入。您可以在此处查看此功能的文档:http://php.net/manual/en/mysqli.real-escape-string.php
答案 1 :(得分:0)
您的代码是正确的。只需要添加()arround查询字符串
Akso你需要从查询字符串结束中删除;
。删除其他条件
else{
$queryString .= ";";
}
更改您的查询,如:
$query = "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES (" . $queryString . ")";
同样建议使用预备语句来防止sql注入
答案 2 :(得分:0)
您需要在获取数据之后和执行之前绑定数据...
$loopLength = 1;
for($v = 0; $v < $loopLength; $v++){
$grabDataResults = grabPageData($URLs[$v], $v);
mysqli_stmt_bind_param($stmt, 'ssssss', $grabDataResults["url"],
$grabDataResults["county"], $grabDataResults["paperco"],
$grabDataResults["date"], $grabDataResults["notice"],
$grabDataResults["id"]);
mysqli_stmt_execute($stmt);
printf("%d Row inserted.\n", mysqli_stmt_affected_rows($stmt));
printf("Error:\n", mysqli_stmt_error($stmt));
echo "(" . $grabDataResults["url"] . "," . $grabDataResults["county"] . "," . $grabDataResults["paperco"] . "," . $grabDataResults["date"] . "," . $grabDataResults["notice"] . "," . $grabDataResults["id"] . ")";
}