Question

下面我有一个循环遍历数组的Php代码，每个代码检查数据库中是否已存在该值，如果没有，则创建它。代码本身正在工作，但循环本身可能非常大，最多可以进行几万次迭代。

如何优化此代码？使用什么以及如何使用。应该有一种更好的方法来插入这么多次而不会遍历每个人。

foreach($arr as $value){
    $checkID = mysqli_query($cenn, "SELECT item_id from items WHERE item_id = '$value'");

    if (!$checkID) {
        die("Query '$checkID' failed to execute for some reason");
    }else{

        if (mysqli_num_rows($checkID) > 0) {
            $user = mysqli_fetch_array($checkID);
            echo "item_id" . checkID . "exists already";
        }
        else{
            echo "item_id: '$user_id' doesn't exist<br>";
            $gw2Api = file_get_contents("https://api.guildwars2.com/v2/items/" . $user_id); //12452 30704
            $gw2Api_result = json_decode($gw2Api,true);

            /*Here would be some code to determine values that are being inserted*/

            if (!array_key_exists("description",$gw2Api_result)) {
                $description = 'No description available...';
            } else{
                if($gw2Api_result['description'] === ''){
                    $description = "No description available...";
                } else {
                    $description = $gw2Api_result['description'];
                }
            }

            $insertItem = "INSERT INTO items 
                             (item_id, name, description, 
                              AccountBindOnUse, AccountBound, 
                              last_update
                             ) 
                      VALUES ('$user_id', '$gw2Api_result[name]', '$description', 
                              '$AccountBindOnUse', '$AccountBound', CURRENT_TIMESTAMP)";


            if ($cenn->query($insertItem) === true) {
                echo "New record '$user_id' created successfully";
            } else {
                echo "Error: " . $sql . "<br>" . $cenn->error;
            }
        }
    }
} // end foreach

问题：如何尽快将许多值，新行插入mysqli数据库。

Answer 1

只需使用批量插入。收集所有行以进行插入，并在一次查询中传递它。

echo 'hi';
if (!empty($arr)) {
    echo 'ok';
    $values             = "'" . implode("', '", $arr) . "'";
    $qExistingItemIds   = mysqli_query($cenn, "SELECT item_id from items WHERE item_id IN($values)");
    $existingItemIds    = [];
    while ($existingItemId = mysqli_fetch_array($qExistingItemIds)) {
        $existingItemIds[] = $existingItemId['item_id'];
    }

    $arr = array_diff($arr, $existingItemIds);

    $inserts = array();

    $i = 0;
    $ic = count($arr);
    foreach ($arr as $value) {
        $i++;
        echo "item_id: $value doesn't exist<br>";

        $gw2Api = file_get_contents("https://api.guildwars2.com/v2/items/" . $value); //12452 30704
        $gw2Api_result = json_decode($gw2Api,true);

        /*Here would be some code to determine values that are being inserted*/

        if (!array_key_exists("description", $gw2Api_result)) {
            $description = 'No description available...';
        } else {
            if ($gw2Api_result['description'] === '') {
                $description = "No description available...";
            } else {
                $description = $gw2Api_result['description'];
            }
        }

        $inserts[] = "
            ('$value', '$gw2Api_result[name]', '$description', '$AccountBindOnUse', '$AccountBound', CURRENT_TIMESTAMP)
        ";

        if ($i == 50 OR $i == $ic) {
            $inserts = implode(",", $inserts);
            $insert = "
                INSERT INTO items
                    (item_id, name, description, AccountBindOnUse, AccountBound, last_update)
                VALUES
                    $inserts
            ";

            if ($cenn->query($insert) === true) {
                echo 'great';
                echo "New records created successfully";
            } else {
                echo "Error: " . $sql . "<br>" . $cenn->error;
            }

            $ic -= 50;
            $i = 0;
            $inserts = array();         
        }
    }
}

所以现在我们只有2个查询。不是千千万万。

批量插入的详细信息： http://www.geeksengine.com/database/data-manipulation/bulk-insert.php

Answer 2

如果使用预准备语句，则应减少到数据库服务器的往返，并且只编译和优化每个查询一次，而不是Number_of_inputs * 2查询。这应该减少工作量。

我很想知道多少。

$sql = "SELECT item_id from items WHERE item_id = ?";
$db_select = $cenn->prepare($sql);
if ( ! $db_select ) {
    echo $cenn->error;
    exit;
}

$sql_insert = "INSERT INTO items 
                       (item_id, name, description, 
                        AccountBindOnUse, AccountBound, last_update) 
                VALUES (?, ?, ?, ?, ?, CURRENT_TIMESTAMP)";

$db_insert = $cenn->prepare($sql);
if ( ! $db_insert ) {
    echo $cenn->error;
    exit;
}


foreach($arr as $value){
    $db_select->bind_param('i', $value);
    $res = $db_select->execute()
    if ( $res === FALSE ) {
        echo $cenn->error;
        exit;
    }

    if ($db_select->num_rows > 0) {
        // dont bother fetching data we already know all we need to
        $user = $db_select->free();
        echo "item_id $value exists already";
    } else {
        echo "item_id: $value doesn't exist<br>";
        $gw2Api = file_get_contents("https://api.guildwars2.com/v2/items/" . $value); 
        $gw2Api_result = json_decode($gw2Api,true);

        if ( ! array_key_exists("description",$gw2Api_result)
            || $gw2Api_result['description'] === '') {
            $description = 'No description available...';
        } else{
            $description = $gw2Api_result['description'];
        }


        $db_insert->bind_param('issss', $value, $gw2Api_result[name],
                                        $description, $AccountBindOnUse, 
                                        $AccountBound)

        if ($cenn->query($insertItem) === true) {
            echo "New record $value' created successfully";
        } else {
            echo "Error: " . $sql_insert . "<br>" . $cenn->error;
        }
    }
} // end foreach

PHP Mysqli使用大型迭代插入数据库

2 个答案: