Question

我有一个110M行的大表。我想将一些字段复制到一个新表中，这里是我想要做的一个粗略的想法：

    DECLARE l_seenChangesTo DATETIME DEFAULT '1970-01-01 01:01:01';
    DECLARE l_migrationStartTime DATETIME;

    SELECT NOW() into l_migrationStartTime;

    -- See if we've run this migration before and if so, pick up from where we left off...
    IF EXISTS(SELECT seenChangesTo FROM migration_status WHERE client_user = CONCAT('this-migration-script-', user())) THEN
        SELECT seenChangesTo FROM migration_status WHERE client_user = CONCAT('this-migration-script-', user()) INTO l_seenChangesTo;
        SELECT NOW() as LogTime, CONCAT('Picking up from where we left off: ', l_seenChangesTo) as MigrationStatus;
    END IF;

    INSERT IGNORE INTO newTable
        (field1, field2, lastModified)
        SELECT o.column1 AS field1, 
               o.column2 AS field2,
               o.lastModified 
          FROM oldTable o
          WHERE 
                o.lastModified >= l_seenChangesTo AND
                o.lastModified <= l_migrationStartTime;

    INSERT INTO migration_status (client_user,seenChangesTo) 
        VALUES (CONCAT('this-migration-script-', user()), l_migrationStartTime) 
        ON DUPLICATE KEY UPDATE seenChangesTo=l_migrationStartTime;

上下文

CREATE TABLE IF NOT EXISTS `newTable` (
    `field1`     varchar(255) NOT NULL,
    `field2`      tinyint unsigned NOT NULL,
    `lastModified`        datetime NOT NULL,

    PRIMARY KEY (`field1`, `field2`),
    KEY `ix_field1` (`field1`),
    KEY `ix_lastModified` (`lastModified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;


CREATE TABLE IF NOT EXISTS `oldTable` (
    `column1`     varchar(255) NOT NULL,
    `column2`      tinyint unsigned NOT NULL,
    `lastModified`        datetime NOT NULL,

    PRIMARY KEY (`column1`, `column2`),
    KEY `ix_column1` (`column1`),
    KEY `ix_lastModified` (`lastModified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE IF NOT EXISTS `migration_status` (
        `client_user` char(64) NOT NULL,
        `seenChangesTo` char(128) NOT NULL,

        PRIMARY KEY (`client_user`)
    );

注意：我在oldTable中还有一些列。 oldTable和newTable都使用mysql在相同的数据库模式中。

复制一张桌子时的一般策略是什么？我是否应该通过复制说50,000行来以迭代方式执行此迁移。

Answer 1

迭代地执行这样的迁移的插入速度将变得非常慢。为什么不SELECT oldTable INTO OUTFILE，然后是LOAD DATA INFILE？

如何将一个非常大的表复制到MYSQL中的另一个表中？

1 个答案: