加速非常大的mysql连接

时间:2011-11-04 15:28:34

标签: mysql mysql-error-2013

问题:

  1. 10分钟后查询中止
  2. 查询应该更快
  3. 我创建了以下查询。这是几个版本中最快的一个。不幸的是,随着更多的数据,即使这个数据在600秒后输出并出现错误“错误代码:2013。在查询期间丢失了与MySQL服务器的连接”。

        CREATE OR REPLACE VIEW 1 AS
        SELECT `Timeperiod`    AS `Timeperiod`    ,
               "at"            AS `Domain`        ,
               `Content Groups`AS `Content Groups`,
        ...
        FROM   a
    
    UNION ALL
    
    SELECT `Timeperiod`    AS `Timeperiod`    ,
           "com"           AS `Domain`        ,
           `Content Groups`AS `Content Groups`,
           ...
    FROM   b
    
    UNION ALL
    
    SELECT `Timeperiod`    AS `Timeperiod`    ,
           "com"                 AS `Domain`,
           `Content Groups`AS `Content Groups`,
           ...
    FROM   c
    
    UNION ALL
    
    SELECT `Timeperiod`    AS `Timeperiod`    ,
           "fr"                 AS `Domain`,
           `Content Groups`AS `Content Groups`,
           ...
    FROM   d
    
    UNION ALL
    
    SELECT `Timeperiod`    AS `Timeperiod`    ,
           "it"                 AS `Domain`,
           `Content Groups`AS `Content Groups`,
           ...
    FROM   e;
    
    CREATE OR REPLACE VIEW 2 AS
    SELECT `Timeperiod`        AS `Timeperiod`       ,
           `Content Group`     AS `Content Group`    ,
           "at"                AS `Domain`,
           ...
    FROM   f
    
    UNION ALL
    
    SELECT `Timeperiod`        AS `Timeperiod`       ,
           `Content Group`     AS `Content Group`    ,
           "com"               AS `Domain`,
           ...
    FROM g
    
    UNION ALL
    
    SELECT `Timeperiod`        AS `Timeperiod`       ,
           `Content Group`     AS `Content Group`    ,
           "com"               AS `Domain`,
           ...
    FROM h
    
    UNION ALL
    
    SELECT `Timeperiod`        AS `Timeperiod`       ,
           `Content Group`     AS `Content Group`    ,
           "fr"                AS `Domain`,
           ...
    FROM   i
    
    UNION ALL
    
    SELECT `Timeperiod`        AS `Timeperiod`       ,
           `Content Group`     AS `Content Group`    ,
           "it"                AS `Domain`,
           ...    
    FROM   j;
    
    CREATE OR REPLACE VIEW 3 AS
    SELECT CG.`Domain`                                        AS `Domain`             ,
           TP.`TimeperiodAlias`                               AS `Timeperiod`         ,
           CG.`Content Groups`                                AS `Content Group`      ,
           M.`InternalName`                                   AS `Internal Model Name`,
           ...
    FROM   1 CG                      ,
           Timperiods TP             ,
           Models M
    WHERE  CG.`Content Groups` LIKE CONCAT(M.`ContentGroupName`, '%')
    AND    CG.`Timeperiod`        = TP.`Timeperiod`;
    
    CREATE OR REPLACE VIEW 4 AS
    SELECT CGD.`Domain`                                              AS `Domain`        ,
           TP.`TimeperiodAlias`                                      AS `Timeperiod`    ,
           CGD.`Content Group`                                       AS `Content Group`,
           ...
    FROM   2 CGD,
           Timeperiods TP                 ,
           Models M 
    WHERE  CGD.`Content Group` LIKE CONCAT(M.`ContentGroupName`, '%')
    AND    CGD.`Timeperiod`       = TP.`Timeperiod`;
    
    DROP TABLE IF EXISTS 5;
    
    CREATE TABLE IF NOT EXISTS 5
                 (
                              `Domain`     VARCHAR(3) NOT NULL ,
                              `Timeperiod` VARCHAR(30) NOT NULL,
                              `Content Group` varchar(70),
                              `Internal Model Name` VARCHAR(50),
                              ...
                               PRIMARY KEY (`Domain`,`Timeperiod`, `Content Group`)
                 )    
    AS        
    
    SELECT CG.`Domain`              AS `Domain`             ,
           CG.`Timeperiod`          AS `Timeperiod`         ,
           CG.`Content Group`       AS `Content Group`      ,
           CG.`Internal Model Name` AS `Internal Model Name`,
           ...
    FROM   3 CG,
           4 CGD
    WHERE  CG.`Content Group` = CGD.`Content Group`
    AND    CG.`Timeperiod`    = CGD.`Timeperiod`
    AND    CG.`Domain`        = CGD.`Domain`;
    

    这些是步骤的行数:

    1:64763 2:51932

    时间段:36

    模特:15

    3:2706

    4:2172

    这是EXPLAIN:

    '1', 'PRIMARY', 'M', 'ALL', NULL, NULL, NULL, NULL, '15', ''
    '1', 'PRIMARY', 'M', 'index', NULL, 'CGIndex', '242', NULL, '15', 'Using index; Using join buffer'
    '1', 'PRIMARY', '<derived3>', 'ALL', NULL, NULL, NULL, NULL, '9528', 'Using where; Using join buffer'
    '1', 'PRIMARY', 'TP', 'eq_ref', 'PRIMARY', 'PRIMARY', '65', 'CG.Timeperiod', '1', ''
    '1', 'PRIMARY', '<derived9>', 'ALL', NULL, NULL, NULL, NULL, '21226', 'Using where; Using join buffer'
    '1', 'PRIMARY', 'TP', 'eq_ref', 'PRIMARY', 'PRIMARY', '65', 'CGD.Timeperiod', '1', 'Using where'
    '9', 'DERIVED', 'ContentGroupDuration_jMKL35_ALL', 'ALL', NULL, NULL, NULL, NULL, '17794', ''
    '10', 'UNION', 'ContentGroupDurationVisitDuration_k4cZ5M_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    '11', 'UNION', 'ContentGroupDurationVisitDuration_k4cZ5M_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    '12', 'UNION', 'ContentGroupDuration_jMKL35_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    '13', 'UNION', 'ContentGroupDuration_jMKL35_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    NULL, 'UNION RESULT', '<union9,10,11,12,13>', 'ALL', NULL, NULL, NULL, NULL, NULL, ''
    '3', 'DERIVED', 'ContentGroups_fd33ef1_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    '4', 'UNION', 'ContentGroups_fd33ef1_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    '5', 'UNION', 'ContentGroups_fd33ef1_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    '6', 'UNION', 'ContentGroups_fd33ef1_ALL', 'ALL', NULL, NULL, NULL, NULL, '10476', ''
    '7', 'UNION', 'ContentGroups_fd33ef1_ALL', 'ALL', NULL, NULL, NULL, NULL, '1', ''
    NULL, 'UNION RESULT', '<union3,4,5,6,7>', 'ALL', NULL, NULL, NULL, NULL, NULL, ''
    

    有没有人知道如何加固查询和/或如何避免连接中止?

    解决方案:

    问题1:从命令行执行“set wait_timeout = 2147483”(不在sql中)

    问题2:将中间结果存储在临时表中并添加索引。然后执行大型连接。

    最佳

    基督教

1 个答案:

答案 0 :(得分:0)

我想说有两种方法: - 更改非交互式连接的超时(wait_timeout for Mysql) - 或以某种方式优化你的表结构

我过去一直在研究大型商业数据库,你加入的性能更多地与表的索引方式有关,而不是与获取的行数相关。确保正确的表格有正确的密钥,如果可能的话,尽量增加它们。

无论如何,更改wait_timeout,如果允许长时间复杂的查询,连接就不会很快死掉。

要更改超时,请以root用户身份登录mysql: mysql -u root -p,输入密码,并输入以下内容: 设置全局wait_timeout = 2147483

这是Windows上23天的最大值。它可以在Linux发行版上获得更高的成就,但不管怎么说你都不需要那么久。

欢呼声,