如何使用非常大的表以最有效的方式使用SQL

时间:2015-04-18 08:44:38

标签: php mysql pdo

我有一张桌子purchase 类似于(idbuyer_idseller_idamount)的内容,包含20-30百万条记录 我也有一个结构相同的表purchase_archive 和表users_balanceiduser_idbalance

我应该编写一个脚本:

  1. 将记录从purchase移至purchase_archive
  2. 对于每个移动的行users_balance应该更新(如果他是买家,则应减少amount的用户余额,如果他是卖家,则增加相同的金额)
  3. 解决此任务的最佳方法是什么? (PHP + Mysql PDO)

    我的假设是:

    1. 将表引擎设置为InnoDB
    2. 从第一张表中选择1000行的批次
    3. 开始交易(这就是InnoDB的原因)
    4. 每行

      4.1将id存储在数组($ temp)

      4.2使用

      等查询更新余额
      SELECT `amount` FROM `purchase` WHERE `id` = :tid LIMIT 1 INTO @amount;
      UPDATE `users_balance` SET `balance` = CASE 
      WHEN `user_id` = :seller_id THEN `balance` + @amount 
      WHEN `user_id` = :buyer_id THEN `balance` - @amount END 
      WHERE `user_id` IN (:buyer_id, :seller_id);
      
    5. 使用以下查询将行移动到存档中:

          INSERT INTO `purchase_archive` SELECT * FROM `purchase` WHERE `id` IN (".$temp.");
          DELETE QUICK FROM `transactions` WHERE `id` IN (".$temp.");
      
    6. 结束交易

    7. 在循环中重复2-6。

      最长的操作是4.2点,我不知道如何在没有变量的情况下更快地执行它

      有更快的方法吗?

      P.S。抱歉我的英语很糟糕。

1 个答案:

答案 0 :(得分:0)

您可以尝试这样的事情:

update user_balance b 
  inner join (
    select b.user_id, 
           sum(case when p.buyer_id = b.user_id then p.amount else 0 end) bought, 
           sum(case when p.seller_id = b.user_id then p.amount else 0 end) sold
    from purchase p 
      inner join user_balance b 
        on p.buyer_id = b.user_id 
          or p.seller_id = b.user_id 
    group by b.user_id) q 
    on b.user_id = q.user_id 
    set b.amount = b.amount + q.sold - q.bought;

它应该在一个查询中执行所有操作。如果您愿意,可以在内部查询中进一步限制范围。 SQL Fiddle似乎已经失败,所以我无法提供现场演示,但有这样的:

mysql> select * from user_balance;
+---------+--------+
| user_id | amount |
+---------+--------+
|       1 |     50 |
|       2 |     50 |
|       3 |     50 |
|       4 |     50 |
+---------+--------+
4 rows in set (0.00 sec)

mysql> select * from purchase;
+-------------+-----------+----------+--------+
| purchase_id | seller_id | buyer_id | amount |
+-------------+-----------+----------+--------+
|           1 |         1 |        2 |     10 |
|           2 |         3 |        4 |     20 |
|           3 |         4 |        2 |      5 |
|           4 |         1 |        4 |      7 |
|           5 |         3 |        1 |      9 |
+-------------+-----------+----------+--------+
5 rows in set (0.00 sec)

并在查询之后......

mysql> update user_balance b inner join (select b.user_id, sum(case when p.buyer_id = b.user_id then p.amount else 0 end) bought, sum(case when p.seller_id = b.user_id then p.amount else 0 end) sold from purchase p inner join user_balance b on p.buyer_id = b.user_id or p.seller_id = b.user_id group by b.user_id) q on b.user_id = q.user_id set b.amount = b.amount + q.sold - q.bought;
Query OK, 4 rows affected (0.07 sec)
Rows matched: 4  Changed: 4  Warnings: 0

mysql> select * from user_balance;
+---------+--------+
| user_id | amount |
+---------+--------+
|       1 |     58 |
|       2 |     35 |
|       3 |     79 |
|       4 |     28 |
+---------+--------+
4 rows in set (0.00 sec)