Question

以下是表格的创建声明：

CREATE TABLE `inodes_data` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `I_ID` int(11) unsigned NOT NULL DEFAULT '0',
  `Time` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  `Stored` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  `dataIndex` int(11) unsigned DEFAULT NULL,
  `memoryAddress` int(11) DEFAULT NULL,
  `Sens1` float DEFAULT NULL,
  `Sens2` float DEFAULT NULL,
  `Sens3` float DEFAULT NULL,
  `Sens4` float DEFAULT NULL,
  `rawData` char(50) NOT NULL,
  PRIMARY KEY (`id`,`Time`,`I_ID`),
  UNIQUE KEY `i_id_time_idx` (`I_ID`,`Time`),
  KEY `I_ID` (`I_ID`),
  KEY `IX_TIME` (`Time`)
) ENGINE=InnoDB AUTO_INCREMENT=8289060 DEFAULT CHARSET=latin1 AVG_ROW_LENGTH=83
/*!50100 PARTITION BY RANGE (TO_DAYS(Time))
(PARTITION p2011_Prior VALUES LESS THAN (0) ENGINE = InnoDB,
 PARTITION p2011_01 VALUES LESS THAN (734534) ENGINE = InnoDB,
 PARTITION p2011_02 VALUES LESS THAN (734562) ENGINE = InnoDB,
 PARTITION p2011_03 VALUES LESS THAN (734593) ENGINE = InnoDB,
 PARTITION p2011_04 VALUES LESS THAN (734623) ENGINE = InnoDB,
 PARTITION p2011_05 VALUES LESS THAN (734654) ENGINE = InnoDB,
 PARTITION p2011_06 VALUES LESS THAN (734684) ENGINE = InnoDB,
 PARTITION p2011_07 VALUES LESS THAN (734715) ENGINE = InnoDB,
 PARTITION p2011_08 VALUES LESS THAN (734746) ENGINE = InnoDB,
 PARTITION p2011_09 VALUES LESS THAN (734776) ENGINE = InnoDB,
 PARTITION p2011_10 VALUES LESS THAN (734807) ENGINE = InnoDB,
 PARTITION p2011_11 VALUES LESS THAN (734837) ENGINE = InnoDB,
 PARTITION p2011_12 VALUES LESS THAN (734868) ENGINE = InnoDB,
 PARTITION p2012_01 VALUES LESS THAN (734899) ENGINE = InnoDB,
 PARTITION p2012_02 VALUES LESS THAN (734928) ENGINE = InnoDB,
 PARTITION p2012_03 VALUES LESS THAN (734959) ENGINE = InnoDB,
 PARTITION p2012_04 VALUES LESS THAN (734989) ENGINE = InnoDB,
 PARTITION p2012_05 VALUES LESS THAN (735020) ENGINE = InnoDB,
 PARTITION p2012_06 VALUES LESS THAN (735050) ENGINE = InnoDB,
 PARTITION p2012_07 VALUES LESS THAN (735081) ENGINE = InnoDB,
 PARTITION p2012_08 VALUES LESS THAN (735112) ENGINE = InnoDB,
 PARTITION p2012_09 VALUES LESS THAN (735142) ENGINE = InnoDB,
 PARTITION p2012_10 VALUES LESS THAN (735173) ENGINE = InnoDB,
 PARTITION p2012_11 VALUES LESS THAN (735203) ENGINE = InnoDB,
 PARTITION p2012_12 VALUES LESS THAN (735234) ENGINE = InnoDB,
 PARTITION p2013_01 VALUES LESS THAN (735265) ENGINE = InnoDB,
 PARTITION p2013_02 VALUES LESS THAN (735293) ENGINE = InnoDB,
 PARTITION p2013_03 VALUES LESS THAN (735324) ENGINE = InnoDB,
 PARTITION p2013_04 VALUES LESS THAN (735354) ENGINE = InnoDB,
 PARTITION p2013_05 VALUES LESS THAN (735385) ENGINE = InnoDB,
 PARTITION p2013_06 VALUES LESS THAN (735415) ENGINE = InnoDB,
 PARTITION p2013_07 VALUES LESS THAN (735446) ENGINE = InnoDB,
 PARTITION p2013_08 VALUES LESS THAN (735477) ENGINE = InnoDB,
 PARTITION p2013_09 VALUES LESS THAN (735507) ENGINE = InnoDB,
 PARTITION p2013_10 VALUES LESS THAN (735538) ENGINE = InnoDB,
 PARTITION p2013_11 VALUES LESS THAN (735568) ENGINE = InnoDB,
 PARTITION p2013_12 VALUES LESS THAN (735599) ENGINE = InnoDB,
 PARTITION p2014_01 VALUES LESS THAN (735630) ENGINE = InnoDB,
 PARTITION p2014_02 VALUES LESS THAN (735658) ENGINE = InnoDB,
 PARTITION p2014_03 VALUES LESS THAN (735689) ENGINE = InnoDB,
 PARTITION p2014_04 VALUES LESS THAN (735719) ENGINE = InnoDB,
 PARTITION p2014_05 VALUES LESS THAN (735750) ENGINE = InnoDB,
 PARTITION p2014_06 VALUES LESS THAN (735780) ENGINE = InnoDB,
 PARTITION p2014_07 VALUES LESS THAN (735811) ENGINE = InnoDB,
 PARTITION p2014_08 VALUES LESS THAN (735842) ENGINE = InnoDB,
 PARTITION p2014_09 VALUES LESS THAN (735872) ENGINE = InnoDB,
 PARTITION p2014_10 VALUES LESS THAN (735903) ENGINE = InnoDB,
 PARTITION p2014_11 VALUES LESS THAN (735933) ENGINE = InnoDB,
 PARTITION p2014_12 VALUES LESS THAN (735964) ENGINE = InnoDB,
 PARTITION p2015_01 VALUES LESS THAN (735995) ENGINE = InnoDB,
 PARTITION p2015_02 VALUES LESS THAN (736023) ENGINE = InnoDB,
 PARTITION p2015_03 VALUES LESS THAN (736054) ENGINE = InnoDB,
 PARTITION p2015_04 VALUES LESS THAN (736084) ENGINE = InnoDB,
 PARTITION p2015_05 VALUES LESS THAN (736115) ENGINE = InnoDB,
 PARTITION p2015_06 VALUES LESS THAN (736145) ENGINE = InnoDB,
 PARTITION p2015_07 VALUES LESS THAN (736176) ENGINE = InnoDB,
 PARTITION p2015_08 VALUES LESS THAN (736207) ENGINE = InnoDB,
 PARTITION p2015_09 VALUES LESS THAN (736237) ENGINE = InnoDB,
 PARTITION p2015_10 VALUES LESS THAN (736268) ENGINE = InnoDB,
 PARTITION p2015_11 VALUES LESS THAN (736298) ENGINE = InnoDB,
 PARTITION p2015_12 VALUES LESS THAN (736329) ENGINE = InnoDB,
 PARTITION p2016_01 VALUES LESS THAN (736360) ENGINE = InnoDB,
 PARTITION p2016_02 VALUES LESS THAN (736389) ENGINE = InnoDB,
 PARTITION p2016_03 VALUES LESS THAN (736420) ENGINE = InnoDB,
 PARTITION p2016_04 VALUES LESS THAN (736450) ENGINE = InnoDB,
 PARTITION p2016_05 VALUES LESS THAN (736481) ENGINE = InnoDB,
 PARTITION p2016_06 VALUES LESS THAN (736511) ENGINE = InnoDB,
 PARTITION p2016_07 VALUES LESS THAN (736542) ENGINE = InnoDB,
 PARTITION p2016_08 VALUES LESS THAN (736573) ENGINE = InnoDB,
 PARTITION p2016_09 VALUES LESS THAN (736603) ENGINE = InnoDB,
 PARTITION p2016_10 VALUES LESS THAN (736634) ENGINE = InnoDB,
 PARTITION p2016_11 VALUES LESS THAN (736664) ENGINE = InnoDB,
 PARTITION p2016_12 VALUES LESS THAN (736695) ENGINE = InnoDB,
 PARTITION p2017_01 VALUES LESS THAN (736726) ENGINE = InnoDB,
 PARTITION p2017_02 VALUES LESS THAN (736754) ENGINE = InnoDB,
 PARTITION p2017_03 VALUES LESS THAN (736785) ENGINE = InnoDB,
 PARTITION p2017_04 VALUES LESS THAN (736815) ENGINE = InnoDB,
 PARTITION p2017_05 VALUES LESS THAN (736846) ENGINE = InnoDB,
 PARTITION p2017_06 VALUES LESS THAN (736876) ENGINE = InnoDB,
 PARTITION p2017_07 VALUES LESS THAN (736907) ENGINE = InnoDB,
 PARTITION p2017_08 VALUES LESS THAN (736938) ENGINE = InnoDB,
 PARTITION p2017_09 VALUES LESS THAN (736968) ENGINE = InnoDB,
 PARTITION p2017_10 VALUES LESS THAN (736999) ENGINE = InnoDB,
 PARTITION p2017_11 VALUES LESS THAN (737029) ENGINE = InnoDB,
 PARTITION p2017_12 VALUES LESS THAN (737060) ENGINE = InnoDB,
 PARTITION p2018_01 VALUES LESS THAN (737091) ENGINE = InnoDB,
 PARTITION p2018_02 VALUES LESS THAN (737119) ENGINE = InnoDB,
 PARTITION p2018_03 VALUES LESS THAN (737150) ENGINE = InnoDB,
 PARTITION p2018_04 VALUES LESS THAN (737180) ENGINE = InnoDB,
 PARTITION p2018_05 VALUES LESS THAN (737211) ENGINE = InnoDB,
 PARTITION p2018_06 VALUES LESS THAN (737241) ENGINE = InnoDB,
 PARTITION p2018_07 VALUES LESS THAN (737272) ENGINE = InnoDB,
 PARTITION p2018_08 VALUES LESS THAN (737303) ENGINE = InnoDB,
 PARTITION p2018_09 VALUES LESS THAN (737333) ENGINE = InnoDB,
 PARTITION p2018_10 VALUES LESS THAN (737364) ENGINE = InnoDB,
 PARTITION p2018_11 VALUES LESS THAN (737394) ENGINE = InnoDB,
 PARTITION p2018_12 VALUES LESS THAN (737425) ENGINE = InnoDB,
 PARTITION p2019_01 VALUES LESS THAN (737456) ENGINE = InnoDB,
 PARTITION p2019_02 VALUES LESS THAN (737484) ENGINE = InnoDB,
 PARTITION p2019_03 VALUES LESS THAN (737515) ENGINE = InnoDB,
 PARTITION p2019_04 VALUES LESS THAN (737545) ENGINE = InnoDB,
 PARTITION p2019_05 VALUES LESS THAN (737576) ENGINE = InnoDB,
 PARTITION p2019_06 VALUES LESS THAN (737606) ENGINE = InnoDB,
 PARTITION p2019_07 VALUES LESS THAN (737637) ENGINE = InnoDB,
 PARTITION p2019_08 VALUES LESS THAN (737668) ENGINE = InnoDB,
 PARTITION p2019_09 VALUES LESS THAN (737698) ENGINE = InnoDB,
 PARTITION p2019_10 VALUES LESS THAN (737729) ENGINE = InnoDB,
 PARTITION p2019_11 VALUES LESS THAN (737759) ENGINE = InnoDB,
 PARTITION p2019_12 VALUES LESS THAN (737790) ENGINE = InnoDB,
 PARTITION pUnknown VALUES LESS THAN MAXVALUE ENGINE = InnoDB) */;

针对此表运行的最常见查询是：

SELECT  *
    FROM  inodes_data
    WHERE  I_ID = xxxxx
      AND  Time BETWEEN 'xxxx-xx-xx xx:xx:xx' AND 'xxxx-xx-xx xx:xx:xx';

或

SELECT  *
    FROM  inodes_data
    WHERE  I_ID IN (xxxxx,xxxx,....)
      AND  Time BETWEEN 'xxxx-xx-xx xx:xx:xx' AND 'xxxx-xx-xx xx:xx:xx';

目前，查询此表中的数据需要很长时间。如果我试着拉一年，可能需要15秒到几分钟。我已经研究过，并且正在努力寻找加速这种方法的方法。有没有更好的方法来设置索引，如果有，有人可以告诉我为什么！？我很感激帮助。

Answer 1

我认为MySQL会使用你的I_ID或Time索引。确保您可以在查询开头使用EXPLAIN来查看MySQL的查询计划实际是什么。（https://dev.mysql.com/doc/refman/5.7/en/explain.html）

ie：EXPLAIN SELECT * FROM inodes_data WHERE I_ID = xxxxx AND Time BETWEEN 'xxxx-xx-xx xx:xx:xx' AND 'yyyy-yy-yy yy:yy:yy'; //添加了AND声明

了解更多信息：EXPLAIN EXTENDED SELECT * FROM inodes_data WHERE I_ID = xxxxx AND Time BETWEEN 'xxxx-xx-xx xx:xx:xx' AND 'yyyy-yy-yy yy:yy:yy'; //添加了AND声明。

另外，您可以创建一个同时包含I_ID和Time的复合索引，如果您始终使用这两列进行查询，则可能会创建更好的索引。您可以更改您的alter以添加另一个索引： - 错过已经是多列索引的现有UNIQUE。

（https://dev.mysql.com/doc/refman/5.7/en/multiple-column-indexes.html）

Answer 2

你真的在做SELECT *吗？这将返回多少行？不知道网络时间是真正的问题吗？

如果你真的在总结一些事情，那就让我们看看GROUP BY等。它可以在如何回答你的问题时产生大的差异。

同时，......

摆脱PK并促使UNIQUE KEY (I_ID,Time)成为PK。这将使第一个查询最佳地工作。事实上，如果没有 PARTITIONing，会更好。

不要超过50个分区。您拥有的分区越多，您支付的开销就越多。

在您需要之前不要添加分区。再次，＆＃39;开销＆＃39;。保持未来＆＃39;分区并在添加新分区时使用REORGANIZE PARTITION。有关详细讨论，请参阅here。

在适用的地方缩小INTs（例如SMALLINT UNSIGNED）。

摆脱多余的KEY (I_ID)。它浪费了磁盘空间并放慢了INSERTs。

摆脱time上的索引 - 使用分区键创建索引几乎总是低效的。

由于第二个查询是在寻求二维索引，我不建议删除分区。这是PARTITION的少数用途之一。

除非文字确实是固定长度，否则不要说CHAR(50)。（不，制作行＆＃34;固定长度＆＃34;没有帮助。）

建议将此构造用于时间范围：

WHERE Time >= '2017-02-01` AND Time < '2017-02-01` + INTERVAL 5 MONTH

要点：

清理分区（更少，没有未来等）

投掷id

CHAR - ＆gt; VARCHAR

将4个索引更改为1：PRIMARY KEY(I_ID, time)

Answer 3

构建并维护两个“摘要表”。每个表都有

Time (truncated to day for one table, hour for the other)
I_ID
3 columns for each sensor
miscellany

一个传感器的列：

average for the day (or hour)
min
max

取决于时间范围...

对于不到一周的范围，请使用原始数据，就像您现在所做的那样。（我们可能需要重新解决分区和索引问题，以使这项工作更好。）
使用每小时汇总表
超过6个月，请使用每日汇总表。

这样，你得到至少100分。调整截止值作为查询速度与图表中细节之间的权衡。

如果您愿意，可以显示一个垂直条而不是一个点 - 这可以让您表示一小时或一天内的变化。如果用户想要更多细节，用户可以放大。

摘要表可以使用

进行非分区

PRIMARY KEY(I_ID, Time),
INDEX(Time)

有关汇总表的更多信息：http://mysql.rjweb.org/doc.php/summarytables

对于表的维护，请执行以下操作：在每个表的末尾，运行如下的查询：

 INSERT INTO Hourly (...)
     SELECT FLOOR(Time / 3600) AS the_hour,
            I_ID, ...
            AVG(sensor1),
            MIN(sensor1),
            MAX(sensor1),
            ...
        FROM ...
        WHERE ...  -- just the one hour
        GROUP BY the_hour, I_ID;

每天结束时，每小时卷入每日。

索引大型mysql表的最佳方法是什么？

3 个答案: