我最初选择了 stock_hfq
表的一部分,然后使用 ts_code
对其进行过滤。
我使用中间表 stock_hfq_temp
来过滤 stock_hfq
数据。查询出来的数据大约有2M行。没有2min 1s
需要exists
,需要1min 5s
需要exists
。但是,加上写临时表stock_hfq_temp
的时间和创建索引的时间{临时表的{1}},总时间ts_code
只有4s。
有没有其他方法可以加快我的查询速度?
difference
在 stock_hfq_temp 表中是唯一的。
相关语句和结果如下:
ts_code
数据库分析如下:
select * from stock_hfq t where t.trade_date>'20110302';
Wall time: 2min 12s
select ts_code from stock_hfq_temp b
Wall time: 8 ms
select * from stock_hfq t where t.trade_date>'20110302' and exists (select 1 from stock_hfq_temp b where b.ts_code=t.ts_code);
Wall time: 1min 5s
两个表都有索引。
mysql> select count(1) from stock_hfq;
+----------+
| count(1) |
+----------+
| 11546271 |
+----------+
1 row in set (3 min 31.64 sec)
mysql> select count(1) from (select distinct ts_code from stock_hfq b) t;
+----------+
| count(1) |
+----------+
| 4480 |
+----------+
1 row in set (1.26 sec)
mysql> select count(1) from stock_hfq_temp;
+----------+
| count(1) |
+----------+
| 1502 |
+----------+
1 row in set (0.18 sec)
解释:
mysql> show index from stock_hfq;
+-----------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
+-----------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| stock_hfq | 1 | ts_code | 1 | ts_code | A | 16782 | NULL | NULL | YES | BTREE | | | YES | NULL |
| stock_hfq | 1 | trade_date | 1 | trade_date | A | 94773 | NULL | NULL | YES | BTREE | | | YES | NULL |
+-----------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
2 rows in set (0.00 sec)
mysql> show index from stock_hfq_temp;
+----------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression |
+----------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
| stock_hfq_temp | 1 | ts_code | 1 | ts_code | A | 1502 | NULL | NULL | YES | BTREE | | | YES | NULL |
+----------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+
1 row in set (0.00 sec)
为所有相关表创建 TABLE 语句:
mysql> explain select * from stock_hfq t where exists (select 1 from stock_hfq_temp b where b.ts_code =t.ts_code);
+----+-------------+-------+------------+-------+---------------+---------+---------+----------------------+------+----------+-------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+----------------------+------+----------+-------------------------------------+
| 1 | SIMPLE | b | NULL | index | ts_code | ts_code | 83 | NULL | 1502 | 100.00 | Using where; Using index; LooseScan |
| 1 | SIMPLE | t | NULL | ref | ts_code | ts_code | 83 | quant_test.b.ts_code | 681 | 100.00 | NULL |
+----+-------------+-------+------------+-------+---------------+---------+---------+----------------------+------+----------+-------------------------------------+
2 rows in set, 2 warnings (0.00 sec)
mysql> show warnings;
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note | 1276 | Field or reference 'quant_test.t.ts_code' of SELECT #2 was resolved in SELECT #1 |
| Note | 1003 | /* select#1 */ select `quant_test`.`t`.`ts_code` AS `ts_code`,`quant_test`.`t`.`trade_date` AS `trade_date`,`quant_test`.`t`.`open` AS `open`,`quant_test`.`t`.`high` AS `high`,`quant_test`.`t`.`low` AS `low`,`quant_test`.`t`.`close` AS `close`,`quant_test`.`t`.`pre_close` AS `pre_close`,`quant_test`.`t`.`change` AS `change`,`quant_test`.`t`.`pct_chg` AS `pct_chg`,`quant_test`.`t`.`vol` AS `vol`,`quant_test`.`t`.`amount` AS `amount`,`quant_test`.`t`.`adj_factor` AS `adj_factor` from `quant_test`.`stock_hfq` `t` semi join (`quant_test`.`stock_hfq_temp` `b`) where (`quant_test`.`t`.`ts_code` = `quant_test`.`b`.`ts_code`) |
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)
/*
Navicat Premium Data Transfer
Source Server : localhost_3306
Source Server Type : MySQL
Source Server Version : 80021
Source Host : localhost:3306
Source Schema : quant_test
Target Server Type : MySQL
Target Server Version : 80021
File Encoding : 65001
Date: 12/06/2021 13:55:28
*/
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for stock_hfq
-- ----------------------------
DROP TABLE IF EXISTS `stock_hfq`;
CREATE TABLE `stock_hfq` (
`ts_code` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci DEFAULT NULL,
`trade_date` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci DEFAULT NULL,
`open` double DEFAULT NULL,
`high` double DEFAULT NULL,
`low` double DEFAULT NULL,
`close` double DEFAULT NULL,
`pre_close` double DEFAULT NULL,
`change` double DEFAULT NULL,
`pct_chg` double DEFAULT NULL,
`vol` double DEFAULT NULL,
`amount` double DEFAULT NULL,
`adj_factor` double DEFAULT NULL,
INDEX `ts_code`(`ts_code`) USING BTREE,
INDEX `trade_date`(`trade_date`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_0900_ai_ci ROW_FORMAT = Dynamic;
SET FOREIGN_KEY_CHECKS = 1;