注意:这涉及ColumnStore。
在工作中,我们有一条很大的SQL语句,该语句占用太多内存才能在prod上执行。我目前正在努力减少查询消耗的大小。我尝试使用不同的方法,但是由于某种原因,到目前为止,除了class sunucular extends Model
{
protected $table = 'sunucular';
}
之外,没有任何解决方法。但是,我需要将其与WITH ... AS (...)
结合使用。
这是我要开始工作的代码
INSERT INTO ...
TRUNCATE db1.myTable;
INSERT INTO db1.myTable(`all`, `needed`, `columns`)
(WITH everything AS (
SELECT all, needed, columns
FROM db1.mainTable T1
JOIN db1.secondTable T2
ON (T1.someCol = T2.someCol)
JOIN db2.thirdTable T3
ON (T1.anotherCol = T3.anotherCol)
LEFT JOIN db1.fourthTable T4
ON (T4.anotherCol = T1.anotherCol)
WHERE T2.yetAnotherCol >= (some_SELECT_subquery)
AND T1.valid = 1
) SELECT * FROM everything);
返回
EXPLAIN (WITH everything AS ...
如果我仅使用+------+-------------+-----------------------+------+---------------+------+---------+------+------+-------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------+------+---------------+------+---------+------+------+-------------------------------------------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 16000000000000 | |
| 2 | PRIMARY | T1 | ALL | NULL | NULL | NULL | NULL | 2000 | Using where with pushed condition |
| 2 | PRIMARY | T2 | ALL | NULL | NULL | NULL | NULL | 2000 | Using where; Using join buffer (flat, BNL join) |
| 2 | PRIMARY | T3 | ALL | NULL | NULL | NULL | NULL | 2000 | Using where; Using join buffer (flat, BNL join) |
| 2 | PRIMARY | T4 | ALL | NULL | NULL | NULL | NULL | 2000 | Using where |
| 3 | SUBQUERY | some_SELECT_subquery | ALL | NULL | NULL | NULL | NULL | 2000 | Using where with pushed condition |
+------+-------------+-----------------------+------+---------------+------+---------+------+------+-------------------------------------------------+
5 rows in set (0,21 sec)
语句,则可以使其起作用。与之类似,我不使用WITH
。完全没有问题,而且这种查询方式甚至更快。我还尝试将查询分为几个INSERT INTO
来进行快速测试,但是由于我认为语法弄乱了,所以放弃了。我对SQL不太满意,对WITH
s(初级开发人员)则不太满意。
当我将JOIN
语句与WITH
合并时,MariaDB会以INSER INTO ...
进行响应。我还尝试过在ERROR 1064 (42000) at line 3: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') SELECT * FROM everything)' at line 1
之后添加分号,合并最后两行,将... valid = 1
之后的开括号放在新行上,还有一些我可能想到的与语法相关的问题。没有运气。
我目前的想法是不能将... AS
与INSERT INTO ... SELECT ...
结合使用。至少在开始应有SELECT的地方没有WITH ...
。这就是我可以从docs那里收集的信息。
因此,简而言之,我的问题是:是否可以将WITH
与INSERT INTO ... SELECT
声明完全结合起来?如果没有,我可以用另一种技术达到类似的效果吗?
我还有其他方法可以提高查询的内存利用率吗?我宁愿不乱使用MariaDB或Docker的配置选项,但是如果那是唯一的可能性,我会考虑的。
答案 0 :(得分:0)
您尝试过吗?
TRUNCATE db1.myTable;
WITH everything AS (
SELECT all, needed, columns
FROM db1.mainTable T1
JOIN db1.secondTable T2
ON (T1.someCol = T2.someCol)
JOIN db2.thirdTable T3
ON (T1.anotherCol = T3.anotherCol)
LEFT JOIN db1.fourthTable T4
ON (T4.anotherCol = T1.anotherCol)
WHERE T2.yetAnotherCol >= (some_SELECT_subquery)
AND T1.valid = 1
) INSERT INTO db1.myTable SELECT * FROM everything;
答案 1 :(得分:0)
尽管我没有找到原始问题的答案,但我们还是决定通过减少子查询中收集的数据量来解决该问题。我没有在原始问题中透露此内容,因为这不是我在发布问题时意识到的解决方案。我们只需要从Python脚本中调用SQL,就可以在该脚本中循环获取想要获取的周数。
WHERE T2.ID >= (SELECT ID - {week_number} FROM db1.secondTable WHERE NOW() BETWEEN monday AND sunday) AND T1.valid = 1);