Question

我有一个表，用于存储我的IP摄像机拍摄的快照数据。通常，摄像机每分钟拍摄多张快照，但我的一台摄像机配置为每分钟只拍摄一张快照。

我从表（和磁盘）中清除项目，但根据以下规则保留：

前7天，所有图片都保存
任何超过7天的事情，只保留当天每小时的第一个快照
任何超过4周的事情，只需在当天的第06,12和18小时保留第一张快照
任何超过3个月的东西，只需在当天的第12小时保留第一张快照。

以下是我当前的查询，它可以正常运行，但它会保留在任何一小时的第一分钟内拍摄的所有快照。

SELECT camera_id,
       timestamp,
       frame,
       filename
FROM snapshot_frame
WHERE ((timestamp < subdate(now(), INTERVAL 7 DAY)
        AND minute(timestamp) != 0)
       OR (timestamp < subdate(now(), INTERVAL 4 WEEK)
           AND (hour(timestamp) NOT IN (6,
                                        12,
                                        18)
                OR minute(timestamp) != 0))
       OR (timestamp < subdate(now(), INTERVAL 3 MONTH)
           AND (hour(timestamp) != 12
                OR minute(timestamp) != 0)))

根据上述规则，如何保留7天以上任何时间戳的每分钟第一张快照？

如果有帮助，表/索引结构：

mysql> describe snapshot_frame;
+-----------+--------------+------+-----+---------+-------+
| Field     | Type         | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| camera_id | int(11)      | NO   |     | NULL    |       |
| timestamp | datetime     | NO   | MUL | NULL    |       |
| frame     | int(11)      | YES  |     | NULL    |       |
| filename  | varchar(100) | YES  | UNI | NULL    |       |
+-----------+--------------+------+-----+---------+-------+
4 rows in set (0.04 sec)

mysql> show index from snapshot_frame;
+----------------+------------+-----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table          | Non_unique | Key_name        | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+-----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| snapshot_frame |          0 | filename        |            1 | filename    | A         |     3052545 |     NULL | NULL   | YES  | BTREE      |         |               |
| snapshot_frame |          1 | idx_time_camera |            1 | timestamp   | A         |     3052545 |     NULL | NULL   |      | BTREE      |         |               |
| snapshot_frame |          1 | idx_time_camera |            2 | camera_id   | A         |     3052545 |     NULL | NULL   |      | BTREE      |         |               |
+----------------+------------+-----------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
3 rows in set (0.42 sec)

mysql> select count(*) from snapshot_frame;
+----------+
| count(*) |
+----------+
|  3030214 |
+----------+
1 row in set (18.47 sec)

更新：所以我设法创建了一个查询，根据我的规则提供了我想要保留的所有快照：

SELECT camera_id,
       TIMESTAMP,
       frame,
       filename
FROM snapshot_frame
WHERE TIMESTAMP >= subdate(now(), INTERVAL 7 DAY)
UNION
  (SELECT camera_id,
          TIMESTAMP,
          frame,
          filename
   FROM snapshot_frame
   WHERE TIMESTAMP < subdate(now(), INTERVAL 7 DAY)
     AND TIMESTAMP >= subdate(now(), INTERVAL 4 WEEK)
     AND minute(TIMESTAMP) = 0
   GROUP BY camera_id,
            year(TIMESTAMP),
            month(TIMESTAMP),
            date(TIMESTAMP),
            hour(TIMESTAMP),
            minute(TIMESTAMP))
UNION
  (SELECT camera_id,
          TIMESTAMP,
          frame,
          filename
   FROM snapshot_frame
   WHERE TIMESTAMP < subdate(now(), INTERVAL 4 WEEK)
     AND TIMESTAMP >= subdate(now(), INTERVAL 3 MONTH)
     AND hour(TIMESTAMP) IN (6,
                             12,
                             18)
     AND minute(TIMESTAMP) = 0
   GROUP BY camera_id,
            year(TIMESTAMP),
            month(TIMESTAMP),
            date(TIMESTAMP),
            hour(TIMESTAMP),
            minute(TIMESTAMP))
UNION
  (SELECT camera_id,
          TIMESTAMP,
          frame,
          filename
   FROM snapshot_frame
   WHERE TIMESTAMP < subdate(now(), INTERVAL 3 MONTH)
     AND hour(TIMESTAMP) = 12
     AND minute(TIMESTAMP) = 0
   GROUP BY camera_id,
            year(TIMESTAMP),
            month(TIMESTAMP),
            date(TIMESTAMP),
            hour(TIMESTAMP),
            minute(TIMESTAMP))

我现在只想弄清楚如何逆转，所以我返回一个结果集，其中包含snapshot_frame中上述查询中不包含的所有行。

任何指针？

Answer 1

我现在使用的解决方案是使用我希望保留的行填充临时表：

CREATE
TEMPORARY TABLE IF NOT EXISTS retain_frames (INDEX idx_time_camera (timestamp, camera_id))AS
SELECT camera_id,
       timestamp,
       frame,
       filename
FROM (
        (SELECT camera_id,
                timestamp,
                frame,
                filename
         FROM snapshot_frame a
         WHERE timestamp >= subdate(now(), INTERVAL 7 DAY))
      UNION
        (SELECT camera_id,
                timestamp,
                frame,
                filename
         FROM snapshot_frame b
         WHERE timestamp < subdate(now(), INTERVAL 7 DAY)
           AND timestamp >= subdate(now(), INTERVAL 4 WEEK)
           AND minute(timestamp) = 0
         GROUP BY camera_id,
                  date(timestamp),
                  hour(timestamp),
                  minute(timestamp))
      UNION
        (SELECT camera_id,
                timestamp,
                frame,
                filename
         FROM snapshot_frame c
         WHERE timestamp < subdate(now(), INTERVAL 4 WEEK)
           AND timestamp >= subdate(now(), INTERVAL 3 MONTH)
           AND hour(timestamp) IN (6,
                                   12,
                                   18)
           AND minute(timestamp) = 0
         GROUP BY camera_id,
                  date(timestamp),
                  hour(timestamp),
                  minute(timestamp))
      UNION
        (SELECT camera_id,
                timestamp,
                frame,
                filename
         FROM snapshot_frame d
         WHERE timestamp < subdate(now(), INTERVAL 3 MONTH)
           AND hour(timestamp) = 12
           AND minute(timestamp) = 0
         GROUP BY camera_id,
                  date(timestamp),
                  hour(timestamp),
                  minute(timestamp))) e

然后使用以下查询选择过时的快照：

SELECT camera_id,
       timestamp,
       frame,
       filename
FROM snapshot_frame a
WHERE NOT EXISTS
    (SELECT camera_id,
            timestamp,
            frame,
            filename
     FROM retain_frames b
     WHERE a.camera_id = b.camera_id
       AND a.timestamp = b.timestamp
       AND a.frame = b.frame)

唯一的问题是临时表的创建大约需要2分钟，并且似乎锁定了数据库，导致偶尔的OperationalError: (1205, 'Lock wait timeout exceeded; try restarting transaction')被我的python代码中的另一个试图插入同一个表的线程抛出

mysql时间戳查询，选择除给定分钟的第一个记录以外的所有记录

1 个答案: