Question

我在基本的MySQL上“确定”，但这是“我的头上的方式”！

目标：

导入数据库
更新已更改的产品
删除已删除的商品
快速有效地

数据库表很大，速度是个问题。

没有MyISAM inoDB会更快吗？每个数据库都在一个唯一的表中。

我被赋予这个作为我正在尝试做的起点：

CREATE TABLE `table` LIKE LiveTable
LOAD DATA INFILE..... INTO `table`
UPDATE `table`  SET delete=1; -- Set the delete field to true  because it will not have been updated
UPDATE `table` INNER JOIN`table`ON `LiveTable.ID`=`table.ID`
SET LiveTable.Col1=table.Col1, LiveTable.Col2=table.Col2….. delete=0
INSERT INTO LiveTable(ID,Col1,Col2,…  delete=0)
SELECT ID,Col1,Col2,...FROM `table`
LEFT JOIN LiveTable 
ON table.ID = LiveTable.ID
WHERE LiveTable.ID IS NULL
DELETE FROM LiveTableWHERE delete = 0
EMPTY TABLE `table`

> CREATE TABLE `product_table`   (
>      `programname` VARCHAR(100) NOT NULL,
>      `name`        VARCHAR(160) NOT NULL,
>      `keywords`    VARCHAR(300) NOT NULL,
>      `description` TEXT NOT NULL,
>      `sku`         VARCHAR(100) NOT NULL,
>      -- This is the only "unique identifier given, none will be duplicates"
>      `price`       DECIMAL(10, 2) NOT NULL,
>      `created`     TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
>      `updatedat`   TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
>      `delete`      TINYINT(4) NOT NULL DEFAULT '0',
>      PRIMARY KEY (`sku`)   ) ENGINE=myisam DEFAULT CHARSET=latin1;
> 
> CREATE TABLE IF NOT EXISTS `temptable` LIKE `product_table`;
> 
> TRUNCATE TABLE `temptable`; -- Remove data from temp table if for some
> reason it has data in it.
> 
> LOAD DATA LOW_PRIORITY LOCAL INFILE "catalog.csv" INTO TABLE
> `temptable`  FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY """"
> LINES TERMINATED BY "\n"  IGNORE 1 LINES (`PROGRAMNAME`, `NAME`,
> `KEYWORDS`, `DESCRIPTION`, `SKU`, `PRICE`);
> 
> 
> UPDATE `temptable` SET    `delete` = 1; -- Set the delete field to
> true UPDATE `temptable` ttable
>        INNER JOIN `product_table` mtable
>          ON ( mtable.sku = ttable.sku ) SET    mtable.programname = ttable.programname,
>        mtable.name = ttable.name,
>        mtable.keywords = ttable.keywords,
>        mtable.description = ttable.description,
>        mtable.sku = ttable.sku,
>        mtable.price = ttable.price,
>        mtable.created = ttable.created,
>        mtable.updatedat = NOW(),-- Set Last Update
>        mtable.delete = 0; -- Set Delete to NO
> 
> -- Not sure what this is for...  I'm LOST at this part...   
> INSERT INTO `product_table` VALUES      (`programname`,
>              `name`,
>              `keywords`,
>              `description`,
>              `sku`,
>              `price`,
>              `created`,
>              `updatedat`,
>              `delete`);
> 
> -- This type of join requires alias as far as I know? 
> SELECT `programname`,
>        `name`,
>        `keywords`,
>        `description`,
>        `sku`,
>        `price`,
>        `created`,
>        `updatedat`,
>        `delete` FROM   `temptable` tmptable
>        LEFT JOIN `product_table` maintbl
>          ON tmptable.sku = maintbl.sku WHERE  maintbl.sku IS NULL;
> 
> DELETE FROM `product_table` WHERE  `delete` = 0;
> 
> TRUNCATE TABLE `temptable`; `` remove all the data from temporary
> table.

Answer 1

我自己在这里回答了这个问题： https://dba.stackexchange.com/questions/16197/innodb-update-slow-need-a-better-option/16283#16283

使用我从这里收到的信息，网络和几个互联网聊天室，我想出了。网络来源：http://www.softwareprojects.com/resources/programming/t-how-to-use-mysql-fast-load-data-for-updates-1753.html

[DEMO][1] http://sqlfiddle.com/#!2/4ebe0/1

过程是：

Import into a new temp table.
Update The old table information with information in Temp table.
Insert new data into the table. (Real world I'm making a new CSV file and using LOAD INTO for the insert)
delete everything that is no longer in the data feed.
delete the temp table.

这似乎是迄今为止最快的过程。

让我知道你的意见。

Answer 2

当INSERT，UPDATE和DELETE正在发生时，InnoDB通常比MyISAM好得多，因为InnoDB使用行级锁定进行更新，而MyISAM使用表级锁定。

这是第一步。

第二步是在使用ALTER TABLE .. DISABLE KEYS将数据加载到表中之前禁用表上的所有索引，然后在使用ALTER TABLE .. ENABLE KEYS加载后启用它们。

以上两项显示您的表现有很大改善。

作为另一种优化，在进行大规模更新时，将它们分解为批次（可能基于主键），以便不会同时锁定所有行。

如何导入数据库，更新已更改的产品，删除已删除的产品

2 个答案: