Question

我有大约4000万行（GPS跟踪器位置）的巨大桌子，每10秒从公司内部的多个设备记录。我想只选择每分钟的第一行，所以我使用group by。问题是桌子每10秒就会成长一次，我几乎尝试了所有的东西，用Google搜索了好几个小时。所以我决定问一个问题。

我使用MySQL 5.7.11 InnoDB池50GB，服务器是Xeon X5650，内存为64GB。

表格结构：

CREATE TABLE `eventData` (
  `id` bigint(20) NOT NULL,
  `position` point NOT NULL,
  `speed` decimal(6,2) DEFAULT NULL,
  `time` datetime DEFAULT NULL,
  `device_id` int(9) DEFAULT NULL,
  `processed` tinyint(1) NOT NULL DEFAULT '0',
  `time_m` datetime GENERATED ALWAYS AS ((`time` - interval second(`time`) second)) VIRTUAL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_czech_ci ROW_FORMAT=DYNAMIC;

ALTER TABLE `eventData`
  ADD PRIMARY KEY (`id`),
  ADD KEY `time` (`time`),
  ADD KEY `device_id` (`device_id`,`processed`),
  ADD KEY `time_m` (`time_m`);

SQL：

SELECT e.time, e.time_m, X(e.position) AS lat, Y(e.position) AS lng
FROM eventData AS e
WHERE
  e.device_id = 86 AND
  e.time BETWEEN '2016-02-29' AND '2016-03-06'
  GROUP BY DAY(e.time),HOUR(e.time),MINUTE(e.time);

说明：

EXPLAIN SELECT e.time, e.time_m, X(e.position) AS lat, Y(e.position) AS lng FROM eventData AS e WHERE   e.device_id = 86 AND   e.time BETWEEN '2016-02-29' AND '2016-03-06'   GROUP BY DAY(e.time),HOUR(e.time),MINUTE(e.time);
+----+-------------+-------+------------+------+----------------+-----------+---------+-------+---------+----------+---------------------------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys  | key       | key_len | ref   | rows    | filtered | Extra                                                               |
+----+-------------+-------+------------+------+----------------+-----------+---------+-------+---------+----------+---------------------------------------------------------------------+
|  1 | SIMPLE      | e     | NULL       | ref  | time,device_id | device_id | 5       | const | 2122632 |     6.40 | Using index condition; Using where; Using temporary; Using filesort |
+----+-------------+-------+------------+------+----------------+-----------+---------+-------+---------+----------+---------------------------------------------------------------------+

描述：

DESCRIBE eventData;
+------------------+------------------------+------+-----+---------+-------------------+
| Field            | Type                   | Null | Key | Default | Extra             |
+------------------+------------------------+------+-----+---------+-------------------+
| id               | bigint(20)             | NO   | PRI | NULL    | auto_increment    |
| position         | point                  | NO   |     | NULL    |                   |
| speed            | decimal(6,2)           | YES  |     | NULL    |                   |
| time             | datetime               | YES  | MUL | NULL    |                   |
| device_id        | int(9)                 | YES  | MUL | NULL    |                   |
| processed        | tinyint(1)             | NO   |     | 0       |                   |
| time_m           | datetime               | YES  | MUL | NULL    | VIRTUAL GENERATED |
+------------------+------------------------+------+-----+---------+-------------------+

我试过了：

没有分组：~0.06s
按天，小时，分钟分组：~4.76s
按虚拟列分组（time_m）：~4.92s
分组e.time DIV 500：~5.02s

我需要达到比5秒更好的效果。请帮忙。

Answer 1

您可以对表进行分区。例如按年份。由于索引要小得多，这将大大提高性能。如果您的环境无法做到这一点，请尝试

GROUP BY date_format（e.time，＆＃39;％d％H％i＆＃39;）;

Answer 2

1）您可以尝试复合索引（device_id，time）

2）尝试按虚拟字段分组：

   div{
        background-color: #232B2B;
        background-image:url('../web/css/cave-background.png');
        background-position:top;
        background-repeat:no-repeat;

       }

mysql巨大的表查询优化组由

2 个答案: