Question

我有一个非常大的表（~1 000 000行）和带有联合，联接和where语句的复杂查询（用户可以选择不同的ORDER BY列和方向）。我需要获得分页行数。如果我在不计算行的情况下运行查询，它会很快完成。如何以最快的方式实现分页？我尝试使用EXPLAIN SELECT和SHOW TABLE STATUS来获得近似行数，但它与实际行数非常不同。我的查询就像这个（简称）：

SELECT * FROM (
    (   
        SELECT * FROM table_1 
        LEFT JOIN `table_a` ON table_1.record_id = table_a.id 
        LEFT JOIN `table_b` ON table_a.id = table_b.record_id 
        WHERE table_1.a > 10 AND table_a.b < 500 AND table_b.c = 1 
        ORDER BY x ASC
        LIMIT 0, 10        
    )      
    UNION        
    (   
        SELECT * FROM table_2
        LEFT JOIN `table_a` ON table_2.record_id = table_a.id 
        LEFT JOIN `table_b` ON table_a.id = table_b.record_id 
        WHERE table_2.d < 10 AND table_a.e > 500 AND table_b.f = 1 
        ORDER BY x ASC
        LIMIT 0, 10                                 
    )                 
) tbl ORDER BY x ASC LIMIT 0, 10

查询结果不受限制约为~100 000行，如何以最快的方式获得此近似计数？我的生产查询示例如下：

SELECT SQL_CALC_FOUND_ROWS * FROM (
    (   
        SELECT
          articles_log.id AS log_id, articles_log.source_table,
          articles_log.record_id AS id, articles_log.dat AS view_dat, 
          articles_log.lang AS view_lang, '1' AS view_count, '1' AS unique_view_count,
          articles_log.user_agent, articles_log.ref, articles_log.ip,
          articles_log.ses_id, articles_log.bot, articles_log.source_type, articles_log.link,   
          articles_log.user_country, articles_log.user_platform,
          articles_log.user_os, articles_log.user_browser,                                          
          `contents`.dat AS source_dat, `contents_trans`.header, `contents_trans`.custom_text 
        FROM articles_log 
        INNER JOIN `contents` ON articles_log.record_id = `contents`.id
                             AND articles_log.source_table = 'contents'  
        INNER JOIN `contents_trans` ON `contents`.id = `contents_trans`.record_id
                                   AND `contents_trans`.lang='lv' 
        WHERE articles_log.dat > 0
          AND articles_log.dat >= 1488319200
          AND articles_log.dat <= 1489355999
          AND articles_log.bot = '0'
          AND (articles_log.record_id NOT LIKE '%\_404' AND articles_log.record_id <> '404'
               OR articles_log.source_table <> 'contents') 
    )      
    UNION        
    (   
        SELECT
          articles_log.id AS log_id, articles_log.source_table,
          articles_log.record_id AS id, articles_log.dat AS view_dat, 
          articles_log.lang AS view_lang, '1' AS view_count, '1' AS unique_view_count,
          articles_log.user_agent, articles_log.ref, articles_log.ip,
          articles_log.ses_id, articles_log.bot,
          articles_log.source_type, articles_log.link,   
          articles_log.user_country, articles_log.user_platform,
          articles_log.user_os, articles_log.user_browser,                                          
        `news`.dat AS source_dat, `news_trans`.header, `news_trans`.custom_text 
        FROM articles_log 
        INNER JOIN `news` ON articles_log.record_id = `news`.id
                         AND articles_log.source_table = 'news'  
        INNER JOIN `news_trans` ON `news`.id = `news_trans`.record_id
                         AND `news_trans`.lang='lv' 
        WHERE articles_log.dat > 0 
          AND articles_log.dat >= 1488319200
          AND articles_log.dat <= 1489355999
          AND articles_log.bot = '0'
          AND (articles_log.record_id NOT LIKE '%\_404' AND articles_log.record_id <> '404'
               OR articles_log.source_table <> 'contents') 
    )      
) tbl ORDER BY view_dat ASC LIMIT 0, 10

非常感谢！

Answer 1

如果您可以使用UNION ALL代替UNION（这是UNION DISTINCT的快捷方式） - 换句话说 - 如果您不需要删除重复项，则可以尝试添加两个子查询的计数：

SELECT 
    (   
        SELECT COUNT(*) FROM table_1 
        LEFT JOIN `table_a` ON table_1.record_id = table_a.id 
        LEFT JOIN `table_b` ON table_a.id = table_b.record_id 
        WHERE table_1.a > 10 AND table_a.b < 500 AND table_b.c = 1      
    )      
    +
    (   
        SELECT COUNT(*) FROM table_2
        LEFT JOIN `table_a` ON table_2.record_id = table_a.id 
        LEFT JOIN `table_b` ON table_a.id = table_b.record_id 
        WHERE table_2.d < 10 AND table_a.e > 500 AND table_b.f = 1                              
    ) 
    AS cnt

如果没有ORDER BY且没有UNION，引擎可能不需要创建一个巨大的临时表。

<强>更新

对于原始查询，请尝试以下操作：

仅选择count(*)。
从第一部分（内容）中删除OR articles_log.source_table <> 'contents'因为我们知道它永远不会是真的。
从第二部分删除AND (articles_log.record_id NOT LIKE '%\_404' AND articles_log.record_id <> '404' OR articles_log.source_table <> 'contents')（新闻）因为我们知道它总是正确的，因为OR articles_log.source_table <> 'contents'总是如此。
删除contents和news的联接。您可以使用*_trans

record_id

删除articles_log.dat > 0，因为articles_log.dat >= 1488319200

结果查询：

SELECT (   
    SELECT COUNT(*)
    FROM articles_log 
    INNER JOIN `contents_trans`
      ON `contents_trans`.record_id = articles_log.record_id
      AND `contents_trans`.lang='lv' 
    WHERE articles_log.bot = '0'
      AND articles_log.dat >= 1488319200
      AND articles_log.dat <= 1489355999
      AND articles_log.record_id NOT LIKE '%\_404'
      AND articles_log.record_id <> '404'
) + (   
    SELECT COUNT(*)
    FROM articles_log 
    INNER JOIN `news_trans`
      ON  `news_trans`.record_id = articles_log.record_id
      AND `news_trans`.lang='lv' 
    WHERE articles_log.bot = '0'
      AND articles_log.dat >= 1488319200
      AND articles_log.dat <= 1489355999
) AS cnt

尝试以下索引组合：

articles_log(bot, dat, record_id)
contents_trans(lang, record_id)
news_trans(lang, record_id)

或

contents_trans(lang, record_id)
news_trans(lang, record_id)
articles_log(record_id, bot, dat)

这取决于数据，哪种组合更好。

我可能错了一点，因为我不知道你的数据和业务逻辑。如果是这样，请尝试调整另一个。

Answer 2

您可以按照documentation中的说明使用SQL_CALC_FOUND_ROWS运行查询时获得计算结果：

select SQL_CALC_FOUND_ROWS *
. . .

然后跑步：

select FOUND_ROWS()

但是，第一次运行需要生成所有数据，因此您最多可以获得20行 - 我不认为它在子查询中尊重LIMIT。

考虑到您的查询结构，我想首先考虑优化查询。例如，真的需要UNION（它会导致删除重复的开销）吗？正如评论中指出的那样，你的联接实际上是伪连接为外连接的内连接。索引可能会提高性能。

您可能想要提出另一个问题，提供样本数据和所需结果，以获得有关此类问题的建议。

MySQL行数

2 个答案: