Question

我有以下代码来读取JSON并将结果存储在DDBB中代码有效，但只插入400条记录需要一分多钟如果我打开json，它的加载速度非常快我做错了什么？

    $db = new PDO('', '', '');
    $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    if(tableExists($db, 'locations') == 1)
    {
        $sql=$db->prepare("DROP TABLE locations");
        $sql->execute();
    }
    $sql ="CREATE TABLE `locations` (id INT(6) UNSIGNED AUTO_INCREMENT PRIMARY KEY, evid INT(6) NOT NULL, place VARCHAR(150), country VARCHAR(150), reg_date TIMESTAMP)" ;
    $db->exec($sql);
    $json = file_get_contents('thejson.php');
    $data = array();
    $data = json_decode($json); 
    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);
            $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");
            $stmt->bindValue(':evid', $id, PDO::PARAM_INT);
            $stmt->bindValue(':places', $place, PDO::PARAM_STR);  
            $stmt->bindValue(':country', $country, PDO::PARAM_STR);      
            $stmt->execute();
        }
    }

Answer 1

所以我要尝试的第一件事就是将准备工作移到循环之外，并将其包装在交易中：

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);

            $stmt->bindValue(':evid', $id, PDO::PARAM_INT);
            $stmt->bindValue(':places', $place, PDO::PARAM_STR);  
            $stmt->bindValue(':country', $country, PDO::PARAM_STR);      
            $stmt->execute();
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

您可以做的另一件事是尝试使用bindParam通过引用绑定变量 - 这样您只需要在开始时对每个变量名称调用bindParam一次，然后只覆盖值每次迭代的那些变量并调用execute。

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");
     $id = 0;
     $place = '';
     $country = '';

     $stmt->bindParam(':evid', $id, PDO::PARAM_INT);
     $stmt->bindParam(':places', $place, PDO::PARAM_STR);  
     $stmt->bindParam(':country', $country, PDO::PARAM_STR); 

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);
            $stmt->execute();
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

与此类似，而不是调用bind*，您只需通过execute传递值：

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $params = array(
                ':id' => $object->id,
                ':places' => $object->name,
                ':country' => substr(strrchr($object->name, "-"), 2)
            );

            $stmt->execute($params);
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

我怀疑使用交易可以获得性能提升，但我不知道在切换绑定方法之间会有很多区别。

你最好的选择可能是在@PavanJiwnani建议的单个查询中插入所有记录：

// first we need to compile a structure of only items 
// we will insert with the values properly transformed

$insertData = array_map(function ($object) {
     if (is_object($object)) {
        return array(
            $object->id,
            $object->name,
            substr(strrchr($object->name, "-"), 2)
        );
     } else {
       return false;
     }
}, $data);

// filter out the FALSE values
$insertData = array_filter($insertData);

// get the number of records we have to insert
$nbRecords = count($insertData);

// $records is an array containing a (?,?,?) 
// for each item we want to insert
$records = array_fill(0, $nbRecords, '(?,?,?)');

// now now use sprintf and implode to generate the SQL like:
// INSERT INTO `locations` (evid, place, country) VALUES (?,?,?),(?,?,?),(?,?,?),(?,?,?)
$sql = sprintf(
    'INSERT INTO `locations` (evid, place, country) VALUES %s',   
    implode(',', $records)
);

$stmt = $db->prepare($sql);

// Now we need to flatten our array of insert values as that is what 
// will be expected by execute()
$params = array();
foreach ($insertData as $datum) {
   $params = array_merge($params, $datum);
}

// and finally we attempt to execute
$stmt->execute($params);

Answer 2

尝试以毫秒为单位回显时间戳，以查看运行缓慢的内容。可能执行400个插入查询（包括打开/关闭连接）。

Answer 3

有许多因素会影响数据库的性能，请提供有关数据库系统，PHP版本和相关硬件的详细信息。

瓶颈可能在：

file_get_contents('thejson.php')

如果从远程主机获取JSON内容，即DB正常运行，则网络速度很慢。

您可能还想考虑移动：

$stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

脱离了foreach循环。

php从json插入mysql中的数据运行得太慢了

3 个答案: