所以,我已经被赋予了在MySQL中复制我们目前通过代码处理的功能的任务。
下面的查询效果非常好,可以在40毫秒内恢复245,000行,但只要您使用组或订单触摸它就会超过6秒。
是否有人对索引需要进行哪些更改或者可能如何修改查询以改进索引有任何建议?
由于
select
s.id as sensorid,
s.sensortypeid,
COALESCE(s.pulserate, 1) as pulserate,
COALESCE(s.correctionFactor, 1) as correctionFactor,
ur.id as unitrateid,
COALESCE(ur.priceperkwh, 0) as priceperkwh,
COALESCE(ur.duosCharges, 0) as duosCharges,
IF(t.blnnonunitratecharges, t.nonunitratecharge/48, 0) as nonunitratecost,
IF(t.blnFeedIn, COALESCE(t.feedInRate, 0), 0) as feedInRate,
IF(t.blnRoc, COALESCE(t.rocRate, 0), 0) as rocRate,
from_unixtime(FLOOR(UNIX_TIMESTAMP(srs.dateTimeStamp)/(30*60))*(30*60)) as timeKey
from sensorreadings srs
inner join sensorpoints sp on (sp.id = srs.sensorpointid)
inner join sensors s on (s.id = sp.sensorid)
left join unitrates ur on ur.id = (
select
ur.id
from unitrates ur, tariffs t, companyhubs ch
where
ur.tariffid = t.id and
t.companyid = ch.companyid and
ch.hubid = s.hubid and
t.utilitytypeid = s.utilitytypeid and
(srs.dateTimeStamp between t.startdate and t.enddate) and
((time(srs.dateTimeStamp) between ur.starttime and ur.endtime) and
(ur.dayMask & POW(2, WEEKDAY(srs.dateTimeStamp)) <> 0) and
(ur.monthMask & POW(2, MONTH(srs.dateTimeStamp) - 1) <> 0))
order by
t.startdate desc,
ur.starttime desc
limit 0, 1
)
left join tariffs t on (t.id = ur.tariffid)
where
s.id = 5289
select
s.id as sensorid,
s.sensortypeid,
COALESCE(s.pulserate, 1) as pulserate,
COALESCE(s.correctionFactor, 1) as correctionFactor,
ur.id as unitrateid,
COALESCE(ur.priceperkwh, 0) as priceperkwh,
COALESCE(ur.duosCharges, 0) as duosCharges,
IF(t.blnnonunitratecharges, t.nonunitratecharge/48, 0) as nonunitratecost,
IF(t.blnFeedIn, COALESCE(t.feedInRate, 0), 0) as feedInRate,
IF(t.blnRoc, COALESCE(t.rocRate, 0), 0) as rocRate,
min(srs.reading) as minReading,
avg(srs.reading) as avgReading,
from_unixtime(FLOOR(UNIX_TIMESTAMP(srs.dateTimeStamp)/(30*60))*(30*60)) as timeKey
from sensorreadings srs
inner join sensorpoints sp on (sp.id = srs.sensorpointid)
inner join sensors s on (s.id = sp.sensorid)
left join unitrates ur on ur.id = (
select
ur.id
from unitrates ur, tariffs t, companyhubs ch
where
ur.tariffid = t.id and
t.companyid = ch.companyid and
ch.hubid = s.hubid and
t.utilitytypeid = s.utilitytypeid and
(srs.dateTimeStamp between t.startdate and t.enddate) and
((time(srs.dateTimeStamp) between ur.starttime and ur.endtime) and
(ur.dayMask & POW(2, WEEKDAY(srs.dateTimeStamp)) <> 0) and
(ur.monthMask & POW(2, MONTH(srs.dateTimeStamp) - 1) <> 0))
order by
t.startdate desc,
ur.starttime desc
limit 0, 1
)
left join tariffs t on (t.id = ur.tariffid)
where
s.id = 5289
group by timeKey
order by timeKey desc
架构
CREATE TABLE `sensorreadings` (
`sensorpointid` int(11) NOT NULL DEFAULT '0',
`reading` decimal(15,5) NOT NULL,
`dateTimeStamp` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`sensorpointid`,`dateTimeStamp`),
KEY `sensormetricid` (`sensormetricid`),
KEY `sensorreadings_timestamp` (`dateTimeStamp`,`sensorpointid`),
KEY `sensorpointid` (`sensorpointid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE `sensorpoints` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`sensorid` int(11) DEFAULT NULL,
`hubpointid` int(11) DEFAULT NULL,
`pointlabel` varchar(255) NOT NULL,
`pointhash` varchar(255) NOT NULL,
`target` decimal(10,0) DEFAULT NULL,
`tolerance` decimal(10,0) DEFAULT '0',
`blnlivepoint` int(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `FK_sensorpoints_sensors` (`sensorid`),
CONSTRAINT `FK_sensorpoints_sensors` FOREIGN KEY (`sensorid`) REFERENCES `sensors` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=8824 DEFAULT CHARSET=latin1;
CREATE TABLE `sensors` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`hubid` int(11) DEFAULT NULL,
`sensortypeid` int(11) NOT NULL DEFAULT '5',
`pulserate` decimal(10,6) DEFAULT NULL,
`utilitytypeid` int(11) NOT NULL DEFAULT '1',
`correctionfactor` decimal(10,3) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `FK_sensors_sensortypes` (`sensortypeid`),
KEY `FK_sensors_hubs` (`hubid`),
KEY `FK_sensors_utilitytypes` (`utilitytypeid`),
CONSTRAINT `FK_sensors_hubs` FOREIGN KEY (`hubid`) REFERENCES `hubs` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `FK_sensors_sensortypes` FOREIGN KEY (`sensortypeid`) REFERENCES `sensortypes` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
) ENGINE=InnoDB AUTO_INCREMENT=5503 DEFAULT CHARSET=latin1;
CREATE TABLE `tariffs` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`companyid` int(11) DEFAULT NULL,
`utilitytypeid` int(11) DEFAULT NULL,
`startdate` date NOT NULL,
`enddate` date NOT NULL,
`blnnonunitratecharges` int(1) DEFAULT '0',
`nonunitratecharge` decimal(16,8) DEFAULT '0.00000000',
`blnFeedIn` int(1) DEFAULT '0',
`blnRoc` int(1) DEFAULT '0',
`rocRate` decimal(16,8) DEFAULT '0.00000000',
`feedInRate` decimal(16,8) DEFAULT '0.00000000',
PRIMARY KEY (`id`),
KEY `companyid` (`companyid`,`utilitytypeid`,`startdate`,`enddate`),
KEY `startdate` (`startdate`,`enddate`),
) ENGINE=InnoDB AUTO_INCREMENT=1107 DEFAULT CHARSET=latin1;
CREATE TABLE `unitrates` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`tariffid` int(11) NOT NULL,
`priceperkwh` decimal(16,8) NOT NULL,
`starttime` time NOT NULL,
`endtime` time NOT NULL,
`duoscharges` decimal(10,5) DEFAULT NULL,
`dayMask` int(11) DEFAULT '127',
`monthMask` int(11) DEFAULT '4095',
PRIMARY KEY (`id`),
KEY `FK_unitrates_tariffs` (`tariffid`),
KEY `times` (`starttime`,`endtime`),
KEY `masks` (`dayMask`,`monthMask`),
CONSTRAINT `FK_unitrates_tariffs` FOREIGN KEY (`tariffid`) REFERENCES `tariffs` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
) ENGINE=InnoDB AUTO_INCREMENT=3104 DEFAULT CHARSET=latin1;
解释
没有团体/订购
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
|----|--------------------|-------|--------|---------------------------------|-------------------------|---------|-------------------------------|------|----------------------------------------------|
| 1 | PRIMARY | s | const | PRIMARY | PRIMARY | 4 | const | 1 | NULL |
| 1 | PRIMARY | sp | ref | PRIMARY,FK_sensorpoints_sensors | FK_sensorpoints_sensors | 5 | const | 1 | Using index |
| 1 | PRIMARY | srs | ref | PRIMARY,sensorpointid | PRIMARY | 4 | dbnameprod.sp.id | 211 | Using index |
| 1 | PRIMARY | ur | eq_ref | PRIMARY | PRIMARY | 4 | func | 1 | Using where |
| 1 | PRIMARY | t | eq_ref | PRIMARY | PRIMARY | 4 | dbnameprod.ur.tariffid | 1 | NULL |
| 2 | DEPENDENT SUBQUERY | ch | ref | hubid | hubid | 5 | const | 1 | Using where; Using temporary; Using filesort |
| 2 | DEPENDENT SUBQUERY | t | ref | PRIMARY,companyid,startdate | companyid | 10 | dbnameprod.ch.companyid,const | 1 | Using where; Using index |
| 2 | DEPENDENT SUBQUERY | ur | ref | FK_unitrates_tariffs,times | FK_unitrates_tariffs | 4 | dbnameprod.t.id | 1 | Using where |
订购/分组
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
|----|--------------------|-------|--------|---------------------------------------------------------------|-------------------------|---------|-------------------------------|------|----------------------------------------------|
| 1 | PRIMARY | s | const | PRIMARY | PRIMARY | 4 | const | 1 | Using temporary; Using filesort |
| 1 | PRIMARY | sp | ref | PRIMARY,FK_sensorpoints_sensors | FK_sensorpoints_sensors | 5 | const | 1 | Using index |
| 1 | PRIMARY | srs | ref | PRIMARY,sensormetricid,sensorreadings_timestamp,sensorpointid | PRIMARY | 4 | dbnameprod.sp.id | 211 | Using index |
| 1 | PRIMARY | ur | eq_ref | PRIMARY | PRIMARY | 4 | func | 1 | Using where |
| 1 | PRIMARY | t | eq_ref | PRIMARY | PRIMARY | 4 | dbnameprod.ur.tariffid | 1 | NULL |
| 2 | DEPENDENT SUBQUERY | ch | ref | hubid | hubid | 5 | const | 1 | Using where; Using temporary; Using filesort |
| 2 | DEPENDENT SUBQUERY | t | ref | PRIMARY,companyid,startdate | companyid | 10 | dbnameprod.ch.companyid,const | 1 | Using where; Using index |
| 2 | DEPENDENT SUBQUERY | ur | ref | FK_unitrates_tariffs,times | FK_unitrates_tariffs | 4 | dbnameprod.t.id | 1 | Using where |
答案 0 :(得分:1)
您正在对计算字段timeKey
进行分组和排序,而db在该字段上没有任何索引。
所以db需要在执行group by之前计算所有行,然后执行排序而不使用index cant来加速计算。
建议:在数据库上创建一个时间字段,并为该字段添加索引。
答案 1 :(得分:0)
在调查性能之前,让我们讨论查询被破坏的可能性。
执行GROUP BY
时,所有非汇总SELECT
值都应包含在GROUP BY
中。否则,可以传递任何随机值。
此外,这种模式:
SELECT ..., AVG(a.x)
FROM a
JOIN b ON ...
GROUP BY a.id
通常会导致行数膨胀(由于JOIN
),然后计算累积行数的聚合。添加COUNT(*)
以查看我是否适合您的情况。对于COUNT
,答案可能是公然错误的;对于AVG
,它可能是微妙的错误;对于MIN
,它可能是正确的。最后GROUP BY
缩小了行数。
通常的解决方法是计算聚合而不 JOINs
(我不确定在您的情况下是否可行)。也许像是......
...
JOIN (
SELECT min(srs.reading) as minReading,
avg(srs.reading) as avgReading,
from_unixtime(FLOOR(UNIX_TIMESTAMP(srs.dateTimeStamp)/
(30*60))*(30*60)) as timeKey
FROM srs
GROUP BY timeKey
) AS r
JOIN ...
将date
和time
放在不同的列中通常是一个“坏”的想法。 DATETIME
或TIMESTAMP
更容易与之比较等等(我不清楚您对自己的日期和时间做了什么。)这也可能是性能问题。
3个表格导致一堆JOINing
,使WHERE s.id = 5289
难以转移到srs。您可能需要重新考虑架构作为另一个性能问题。
我意识到价值观不同,但可能
order by
t.startdate desc,
ur.starttime desc
替换为
order by srs.dateTimeStamp
这可能会使我们成为一个指数。
我很惊讶您使用DECIMAL(m,n)
代替FLOAT
来传感器读数。