Question

我用PHP编写了一个用于大学大数据项目的脚本。

以下代码应支持以下功能：

从表中读取数据量
读取同一个表的子集，并用2列扩展它，计算2个值。
生成f.e. 1000个插入并将它们提交到另一个表
对第1点的整个数据集执行此操作

我的问题：

0-1000：我的解决方案插入第一次尝试1000个数据线 1000-2000：在此步骤中，它插入了2000个数据线，因此在第二步之后，它似乎插入了原始数据集的0-2000个数据线。总计3000个数据线。

我有点想法......我没有看到我的失败。

<?php
$user = 'root';
$pass = '';
$kill = 0;
echo 'Connecting to DB:';
try {
    $db1 = new PDO('mysql:host=localhost;dbname=itt2_joined;charset=utf8',$user,$pass);
    $db1->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
    echo "OK \nGOING2COPY!\n";
}
catch(PDOException $ex) {
    echo "  Failed - Process stopped!\n";
    $kill = 1;
}
if(!$kill) {
    $stmt = 'SELECT
                `id_transaction`
              FROM
                `database_joined_full`
              WHERE
                `relative_delivery_date` > 0
              AND
                `relative_received_date` > 0
              AND
                `relative_issued_date` > 0';
    $setSize = $db1->query($stmt)->rowCount();
    echo "No. Of Datasets: " . $setSize . "\n";
    $stepSize = 1000;
    echo "Start Copy $stepSize Datasets per Query! \n";
    $copy = 1;
    $lower = 0;
    $upper = 0;
    echo "[";
    while($copy) {

        $lower = $upper;
        $upper += $stepSize;
        $upper = ($upper > $setSize ? $setSize : $upper);

        $data = $db1->query("SELECT
                `id_transaction`,
                `id_order_line`,
                `relative_transaction_date`,
                `relative_requested_date`,
                `new_requested_qty`, `new_requested_price`,
                `relative_suggested_date`,
                `new_suggested_qty`,
                `delivery_qty`,
                `relative_delivery_date`,
                `received_qty`,
                `relative_received_date`,
                `id_order_header`,
                `delivery_counter`,
                `relative_issued_date`,
                `relative_xcurrent_date`,
                `issued_quantity`,
                `current_quantity`,
                `product_code`,
                `destination`,
                `relative_issuing_date`,
                `delivery_location`,
                `issuer`,
                `id_supplier`,
                `currency`,
                `shipment`,
                `issued_price4unit`,
                `current_price4unit`,
                `total_delivered`
              FROM
                `database_joined_full`
              WHERE
                `relative_delivery_date` > 0
              AND
                `relative_received_date` > 0
              AND
                `relative_issued_date` > 0
              LIMIT  $lower OFFSET $stepSize")->fetchAll();
        $db1->beginTransaction();
        foreach($data as $row) {
           $sql = "";
            $sql = "INSERT INTO `refactored_data` (`id_transaction`, `id_order_line`, `relative_transaction_date`,
                                `relative_requested_date`, `new_requested_qty`, `new_requested_price`, `relative_suggested_date`,
                                `new_suggested_qty`, `delivery_qty`, `relative_delivery_date`, `received_qty`,
                                `relative_received_date`, `id_order_header`, `delivery_counter`, `relative_issued_date`,
                                `relative_xcurrent_date`, `issued_quantity`, `current_quantity`, `product_code`, `destination`,
                                `relative_issuing_date`, `delivery_location`, `issuer`, `id_supplier`, `currency`, `shipment`,
                                `issued_price4unit`, `current_price4unit`, `total_delivered`, `shipment_delay`, `sending_delayed`) VALUES ";
            $sql .= "(";
            for($i = 0; $i<sizeof($row)/2; $i++) {
                $sql .= "'".$row[$i] . "', ";
            }

            $sql .= $row['relative_received_date'] - $row['relative_delivery_date'] . ',';
            $sql .= ($row['relative_received_date'] <= $row['relative_issued_date'] ? 1 : 0) .')';
            $db1->prepare($sql)->execute();

        }
        $db1->commit();


        // Statusbar
        //echo "\n$lower - $upper\n";
        $perc = ceil(($upper/$setSize) *40);
        $bar = "\r[". str_repeat("=", $perc - 1). ">";
        $bar .= str_repeat(" ", 40 - $perc) . "] - " . number_format($upper / $setSize * 100, 3) ."% ";
        echo "\033$bar"; // Note the \033. Put the cursor at the beginning of the line

        //While_Exit
        $copy = ($upper == $setSize ? 0 : 1);
        //$copy = 0;

    }
    echo "\nall done, BYE! \n";
}
?>

Answer 1

您每次都使用步长作为偏移：

LIMIT  $lower OFFSET $stepSize

应该是：

LIMIT $stepSize OFFSET $lower

Answer 2

哦，谢谢！我还没有看到失败。

现在代码工作，几乎完全，我的最后一个问题如下：

在第一个选择查询中我来自＆＃39; - ＆gt; rowCount（）;＆＃39;选择查询的行数。

但是我的插入复制了更多的数据。但是在第二个Select中，我有相同的Where子句和第一个Select查询。

我从CLI-Print获得：行数：655056 它写在数据库658597

中

那么如何获得3541行？

或者插入内部可能是失败？

问候

在pdo中循环时插入失败（pdo / php 5.6）

2 个答案: