Curl Web Scraper问题,错误的数组

时间:2016-01-12 13:21:42

标签: php post curl while-loop screen-scraping

我的刮痧脚本有问题。我喜欢在一段时间内通过id从我的sql中选择值,并将已删除的内容保存在我的数据库中。它的工作原理,但curl脚本每次从第一个请求的内容发送错误的结果到我的数据库。

我的代码:



<?php
error_reporting(E_ALL);
include('db.php');

$i = 1;

while ($i <= 5)
{

$sql = "SELECT * FROM `plz` WHERE `id` = '$i'";
    $row = mysql_fetch_assoc( mysql_query($sql) );

$plz = $row['plz'];


//create array of data to be posted
$post_data['adv_plz'] = "$plz";
$post_data['finda'] = 'adv';
$post_data['lang'] = 'de_DE';

//traverse array and prepare data for posting (key1=value1)
foreach ( $post_data as $key => $value) {
    $post_items[] = $key . '=' . $value;
}

//create the final string to be posted using implode()
$post_string = implode ('&', $post_items);

//create cURL connection
$curl_connection = curl_init('https://www.domain.de/');

//set options
curl_setopt($curl_connection, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($curl_connection, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($curl_connection, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_connection, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl_connection, CURLOPT_FOLLOWLOCATION, 1);

//set data to be posted
curl_setopt($curl_connection, CURLOPT_POSTFIELDS, $post_string);

//perform our request
$result = curl_exec($curl_connection);

//show information regarding the request
$result = utf8_encode($result);

mysql_query("UPDATE plz SET content = '$result' WHERE id = '$i'");


    $i++;
}


?>
&#13;
&#13;
&#13;

这是echo $ post_string的内容

&#13;
&#13;
adv_plz=01000&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE&adv_plz=01002&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE&adv_plz=01002&finda=adv&lang=de_DE&adv_plz=01003&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE&adv_plz=01002&finda=adv&lang=de_DE&adv_plz=01003&finda=adv&lang=de_DE&adv_plz=01004&finda=adv&lang=de_DE
&#13;
&#13;
&#13;

我希望有人可以帮助我。如果我在一段时间内尝试使用该脚本,它可以完美地运行。

1 个答案:

答案 0 :(得分:0)

尝试在进入foreach循环之前重置$ post_items变量。

$post_items = array();

//traverse array and prepare data for posting (key1=value1)
foreach ( $post_d ....