遍历表并使用存储过程或函数

时间：2016-06-15 16:51:38

标签： mysql stored-procedures iteration mysql-workbench do-while

绝望地坚持到目前为止，直到现在我的编程快速拨号伙伴都没有能够提供帮助（大多数人不是MySQL专家）：

我有不同的表，其中使用CSV文件从“导入表数据向导”自动生成列名和数据类型，并且该表还没有包含AUTO INCREMENT列。这个特殊的表由大约30.000行组成。它从一个如下所示的表开始于row = id（1）：

我正在尝试使用一个'更正'表来纠正comma delimited一列中的值。为此，我正在编写一个包含WHILE循环的存储过程，以便通过行的更正表行进行交互，并检查是否在导入的表中找到了别名。

|  id  | material   | alias01   | alias02  | alias03 | *up to 12
    1   Katoen       Cotton      Supima     Pima        
    2   Polyester    Polyster               
    3   Lyocell      Lycocell    Lyocel         
    4   Linnen       Linen              
    5   Viscose      Visose      Viskose    Viscoe  Voscose 
    6   Scheerwol                   
    7   Polyamide                   
    8   Nylon                   
    9   Leer         Leder       Lamsleder  Varkensleder
    10  Polyurethaan Polyurethan PU         Polyuretaan

为了测试目的来测试任何类型的结果我现在只使用alias01（它需要检查alias01，然后是02等...但我会在稍后尝试解决这个问题）。

需要比较`Length'（alias_string_length = found_string_length）以确保在'wool'或'wol'中找不到由'wo'组成的字符串。

需要更正的列中的值看起来像这样（逗号不需要在那里它就是我给予的工作）：

| material |
,Katoen,Elastaan,Voering,Acetaat,Polyester
,Nylon,Polyester,Elastaan

,Katoen
,Leder,in,Leder,Loopzool,Leder
,Polyester
,Polyester,Elastaan,Voering,Polyester

更新

感谢Drew的提示，我改变了程序。我添加了一个tmp表，其中包含材料和每行的唯一ID，并使用alias01迭代每个。执行9000行需要大约11秒，但0 row(s) affected,。关于提高速度的任何提示都是最受欢迎的，但对可能存在的问题的深入了解会有所帮助。

CREATE DEFINER=`root`@`localhost` PROCEDURE `replace_materials`()
BEGIN
set @rownumber = 1;
set @totalrows = 28;
set @um ='';
set @cm ='';
set @corrected ='';
set @correctme ='';

TRUNCATE TABLE tmp;
INSERT INTO tmp (material) SELECT material FROM vantilburgonline.productinfo;


    WHILE (@rownumber < @totalrows) DO

        SET @um = (SELECT alias01 FROM vantilburgonline.materials WHERE id=@rownumber); 
        -- gives 'um' value from column alias01, from table materials, row(X)
        SET @cm = (SELECT material FROM vantilburgonline.materials WHERE id=@rownumber); 
        -- gives 'cm' value from column material, from table materials, row(X)


        set @tmprow = 1;
        set @totaltmprow =9000;

        WHILE (@tmprow < @totaltmprow) DO

            SET @correctme = (SELECT material FROM vantilburgonline.tmp WHERE id = @tmprow); 
            -- gives the value from column material from table tmp to correctme(X).

            SET @correctme = REPLACE(@correctme,@um,@cm);
            -- should run through column material from table productinfo and replace 'alias01' with correct 'material'.
            SET @tmprow = @tmprow +1;

            END WHILE;

        SET @rownumber = @rownumber +1;

    END WHILE;
END

虽然我确定alias01包含它应该在材料中找到的字符串。 Workbench此时也使用了9GB，我只能通过重新启动来解决这个问题。

1 个答案:

答案 0 :(得分：1)

我建议您对materials表进行更改，该表对多列（alias01 .. alias12）不实用。转换为规范化，可扩展的系统。它将有一个材质表和一个material_alias表。当它与您创建的当前表并排放置时，我使用2命名它们。

模式

drop table if exists materials2;
create table materials2
(   material varchar(100) primary key, -- let's go with a natural key
    active bool not null -- turn it LIVE and ON for string replacement of alias back to material name
    -- so active is TRUE for ones to do replacement, or FALSE for skip
    -- facilitates your testing of your synonyms, translations, slangs, etc
)engine=INNODB;

insert materials2 (material,active) values
('KARTON',true),
('Polyester',false),
('Lyocell',false),
('Linnen',true),
('Viscose',true),
('Scheerwol',false),
('Nylon',false),
('Leer',true),
('Polyurethaan',true),
('Polyacryl',true),
('Acryl',false),
('Modal',true),
('Acetaat',true),
('Papier',false),
('Wol',true),
('Zijde',true),
('Temcal',false),
('Polyamide',true),
('Wol-Merino',true),
('Elastan',true),
('Elastomultiester',true);
-- 21 rows
-- a few rows were skipped. The intent of them read as gibberish to me. Please review.

-- we need to restructure the materials2_alias table (after the first attempt)
-- 1. it might need special handling when `alias` is a legitimate substring of `material` (those 2 columns)
-- 2. it needs a unique composite index
drop table if exists materials2_alias;
create table materials2_alias
(   id int auto_increment primary key,
    material varchar(100) not null,
    alias varchar(100) not null,
    ais bool not null, -- Alias is Substring (alias is a legitimate substring of material, like Wo and Wol, respectively)
    unique key(material,alias), -- Composite Index, do not allow dupe combos (only 1 row per combo)
    foreign key `m2alias_m2` (material) references materials2(material)
)engine=INNODB;

insert materials2_alias (material,alias,ais) values
('KARTON','Cotton',false),('KARTON','Katoen',false),('KARTON','Pima',false),
('Polyester','Polyster',false),
('Lyocell','Lycocell',false),('Lyocell','Lyocel',false),
('Linnen','Linen',false),
('Viscose','Visose',false),('Viscose','Viskose',false),('Viscose','Viscoe',false),('Viscose','Voscose',false),
('Leer','Leder',false),('Leer','Lamsleder',false),('Leer','Varkensleder',false),('Leer','Schapenleder',false),('Leer','Geitenleder',false),
('Polyurethaan','Polyurethan',false),('Polyurethaan','PU',false),('Polyurethaan','Polyuretaan',false),('Polyurethaan','Polyurathane',false),('Polyurethaan','Polyurtaan',false),('Polyurethaan','Polyueretaan',false),
('Polyacryl','Polyacrylic',false),
('Acetaat','Leder',false),('Acetaat','Lamsleder',false),
('Wol','Schuurwol',false),('Wol','Wool',false),('Wol','WO',false),('Wol','Scheerwol',false),
('Zijde','Silk',false),('Zijde','Sede',false),
('Polyamide','Polyamie',false),('Polyamide','Polyamid',false),('Polyamide','Poliamide',false),
('Wol-Merino','Merino',false),
('Elastan','Elastaan',false),('Elastan','Spandex',false),('Elastan','Elataan',false),('Elastan','Elastane',false),
('Elastomultiester','elastomutltiester',false),('Elastomultiester','Elasomultiester',false);

-- this cleans up the above, where false should have been true
update materials2_alias
set ais=true 
where instr(material,alias)>0;
-- 4 rows

有几个alter table语句和其他内容。我会尝试记录它们或链接到它们。考虑到你有几百行代码，我只是想捕捉一些东西来分享。但是我的归结为一个简单的代码块，你可以把它放在一个循环中。

更新进入循环： 

UPDATE productinfo pi
join materials2_alias ma 
on instr(  pi.material,  concat(',',ma.alias,',')  )>0 
join materials2 m
on m.material=ma.material and m.active=true
set pi.material=replace(lower(pi.material),lower(ma.alias),lower(ma.material)),
pi.touchCount=pi.touchCount+1;

关于更新的一些注意事项：

-- Note, pi.material starts and ends with a comma.
-- I forced that during the ETL. But `ma.alias` does not contain commas.
-- So add the commas with a concat() within the "Update with a Join" pattern shown
--
-- Note that the commas solved the problem with the Wol - Wo

嗯，特别是以下4个。

select * from materials2_alias 
where ais=true 
order by material,alias;
+----+------------+----------+-----+
| id | material   | alias    | ais |
+----+------------+----------+-----+
|  6 | Lyocell    | Lyocel   |   1 |
| 33 | Polyamide  | Polyamid |   1 |
| 28 | Wol        | WO       |   1 |
| 35 | Wol-Merino | Merino   |   1 |
+----+------------+----------+-----+


-- instr() is not case sensitive except for binary strings
-- REPLACE(str,from_str,to_str); -- case sensitive
-- http://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_replace
--
-- so the update uses lower() or this won't work due to replace() case sensitivity
--

存储过程： 

DROP PROCEDURE if exists touchCounts;
DELIMITER $$
CREATE PROCEDURE touchCounts()
BEGIN
    select touchCount,count(*) as rowCount 
    from productinfo 
    group by touchCount 
    order by touchCount;
END $$
DELIMITER ;

当该存储过程在后续调用（下一次调用）中返回相同的行数时，您已完成通过更新修改material列。

该存储过程可以自然地为rowcount返回out参数。但是现在已经很晚了。

对于您身边的最后一个数据集，需要调用update语句4次。这就像我平庸的笔记本电脑上的13秒。这个想法自然是灵活的，如果你愿意，每种材料可以有数百个别名。

我把它停在github，因为它太多了。

相关问题
最新问题