我现在一直在努力尝试在我的mysql / mariadb数据库中生成自动聚合代码。我目前正在尝试的方法使用变量。我会事先承认,我不是一个数据库专家。我完全是自学成才,一直在努力为这个特殊问题找到足够的资源。我已经在下面添加了简化的示例,哦,我使用mariadb 10.1
。
此代码应该在mysql 5.6
以及mariadb 10.0+
中运行,我已经在10.1上测试了它并且它可以正常工作。
这是我的表格:SQL FIDDLE< - 由于某种原因不起作用。可能是动态列。如果有人知道原因,我会离开它。
CREATE TABLE data_points
(
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
device_id INTEGER,
dtime DATETIME,
sf INTEGER(11), -- sample frequency or interval
agg INTEGER(11), -- aggregation type, actually a fk
data_point BLOB,
PRIMARY KEY (id),
UNIQUE (device_id, dtime, sf, agg)
);
让我们插入一些数据:
INSERT INTO data_points
(device_id, dtime, sf, agg, data_point)
VALUES
(1, '2015-01-02 12:00:00', 1, 60, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 13:00:00', 1, 60, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 14:00:00', 1, 60, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 15:00:00', 1, 60, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 16:00:00', 1, 60, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45));
所以到目前为止一切正常。我试图做的是在不同的时间段内进行聚合,我的最低粒度是60秒。这是我有问题的地方,它可能是显而易见的。
SELECT
@dp_dtime := MAX(dtime),
@dp_aa := MIN(ROUND(COLUMN_GET(data_point, 'aa' AS DOUBLE), 4)),
@dp_ab := MIN(ROUND(COLUMN_GET(data_point, 'ab' AS DOUBLE), 4)),
@dp_ac := MIN(ROUND(COLUMN_GET(data_point, 'ac' AS DOUBLE), 4))
FROM data_points
WHERE
device_id = 1 AND
dtime BETWEEN '2015/01/02 12:00:00' AND '2015/01/17 23:05:00' AND
sf = 60 AND
agg = 1;
INSERT INTO data_points
(device_id, dtime, sf, agg, data_point)
VALUES (8, @dp_dtime, 300, 2, COLUMN_CREATE('aa', @dp_aa, 'ab', @dp_ab, 'ac', @dp_ac));
这最终会在语句中的变量所在的位置创建另一行NULL
。
select @dp_dtime, @dp_aa, @dp_ab, @pd_ac;
-- This results in NULL, NULL, NULL, NULL
此时我很确定我对变量做错了。 这是晚了,14小时的一天。我甚至关闭了吗?有更好/更简单的方法吗? 任何帮助将不胜感激。
修改 在我的实际用例中,列数取决于正在进行聚合的设备类型。列是优秀的风格' aa'通过' zz'可能。虽然我看到的最大值大约是150匝。这可能听起来像一个糟糕的设计,但性能是令人惊讶的,我无法区分这些动态列和实际列之间的区别。 (至少只要你不需要为它们编制索引)
答案 0 :(得分:1)
可能是一个简单的拼写错误:我看到@_dtime
。
在UNIQUE
索引中,将dtime
放在最后;它会使查询更快。 迷你索引课程:所有=
列应该以复合索引排在第一位,任何顺序(基数几乎没有区别)。然后你可以放一个范围' (dtime
)。范围之后的任何列都不用于过滤。请参阅my cookbook。
摆脱id
并将UNIQUE
索引提升为PRIMARY KEY
;它会使查询更快。 迷你索引课程:辅助密钥(例如您的UNIQUE
)需要在密钥和数据之间进行弹跳。 PRIMARY KEY
与数据(在InnoDB中)聚集在一起,从而避免了弹跳。相反,范围扫描'超过PK是一个范围超过表。
答案 1 :(得分:1)
尝试以下查询。
CREATE TABLE data_points
(
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
device_id INTEGER,
dtime DATETIME,
sf INTEGER(11), -- sample frequency or interval
agg INTEGER(11), -- aggregation type, actually a fk
data_point BLOB,
UNIQUE (device_id, dtime, sf, agg)
);
INSERT INTO data_points
(device_id, dtime, sf, agg, data_point)
VALUES
(1, '2015-01-02 12:00:00', 1, 1, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 13:00:00', 1, 1, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 14:00:00', 1, 1, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 15:00:00', 1, 1, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45)),
(1, '2015-01-02 16:00:00', 1, 1, COLUMN_CREATE('aa', 12, 'ab', 34, 'ac', 45));
select * from data_points;
SELECT
@dp_dtime := MAX(dtime) as dp_dtime,
@dp_aa := MIN(ROUND(COLUMN_GET(data_point, 'aa' AS DOUBLE), 4)) as dp_aa,
@dp_ab := MIN(ROUND(COLUMN_GET(data_point, 'ab' AS DOUBLE), 4)) as dp_ab,
@dp_ac := MIN(ROUND(COLUMN_GET(data_point, 'ac' AS DOUBLE), 4)) as dp_ac
FROM data_points
WHERE
device_id = 1 AND
dtime BETWEEN '2015/01/02 12:00:00' AND '2015/1/17 23:05:00' AND
sf = 1 AND
agg = 1;
INSERT INTO data_points
(device_id, dtime, sf, agg, data_point)
VALUES (8, @dp_dtime, 300, 2, COLUMN_CREATE('aa', @dp_aa, 'ab', @dp_ab, 'ac', @dp_ac));
select * from data_points;
mysql> select * from data_points;
+----+-----------+---------------------+------+------+----------------------------+
| id | device_id | dtime | sf | agg | data_point |
+----+-----------+---------------------+------+------+----------------------------+
| 1 | 1 | 2015-01-02 12:00:00 | 1 | 1 | aaabacDZ |
| 2 | 1 | 2015-01-02 13:00:00 | 1 | 1 | aaabacDZ |
| 3 | 1 | 2015-01-02 14:00:00 | 1 | 1 | aaabacDZ |
| 4 | 1 | 2015-01-02 15:00:00 | 1 | 1 | aaabacDZ |
| 5 | 1 | 2015-01-02 16:00:00 | 1 | 1 | aaabacDZ |
+----+-----------+---------------------+------+------+----------------------------+
5 rows in set (0.00 sec)
mysql> SELECT
-> @dp_dtime := MAX(dtime) as dp_dtime,
-> @dp_aa := MIN(ROUND(COLUMN_GET(data_point, 'aa' AS DOUBLE), 4)) as dp_aa,
-> @dp_ab := MIN(ROUND(COLUMN_GET(data_point, 'ab' AS DOUBLE), 4)) as dp_ab,
-> @dp_ac := MIN(ROUND(COLUMN_GET(data_point, 'ac' AS DOUBLE), 4)) as dp_ac
-> FROM data_points
-> WHERE
-> device_id = 1 AND
-> dtime BETWEEN '2015/01/02 12:00:00' AND '2015/1/17 23:05:00' AND
-> sf = 1 AND
-> agg = 1;
+---------------------+---------+---------+---------+
| dp_dtime | dp_aa | dp_ab | dp_ac |
+---------------------+---------+---------+---------+
| 2015-01-02 16:00:00 | 12.0000 | 34.0000 | 45.0000 |
+---------------------+---------+---------+---------+
1 row in set (0.00 sec)
mysql> INSERT INTO data_points
-> (device_id, dtime, sf, agg, data_point)
-> VALUES (8, @dp_dtime, 300, 2, COLUMN_CREATE('aa', @dp_aa, 'ab', @dp_ab, 'ac', @dp_ac));
Query OK, 1 row affected (0.00 sec)
mysql> select * from data_points;
+----+-----------+---------------------+------+------+-------------------------------------------------+
| id | device_id | dtime | sf | agg | data_point |
+----+-----------+---------------------+------+------+-------------------------------------------------+
| 1 | 1 | 2015-01-02 12:00:00 | 1 | 1 | aaabacDZ |
| 2 | 1 | 2015-01-02 13:00:00 | 1 | 1 | aaabacDZ |
| 3 | 1 | 2015-01-02 14:00:00 | 1 | 1 | aaabacDZ |
| 4 | 1 | 2015-01-02 15:00:00 | 1 | 1 | aaabacDZ |
| 5 | 1 | 2015-01-02 16:00:00 | 1 | 1 | aaabacDZ |
| 6 | 8 | 2015-01-02 16:00:00 | 300 | 2 | ▒ aaabac (@ A@ ▒F@ |
+----+-----------+---------------------+------+------+-------------------------------------------------+
6 rows in set (0.00 sec)