我正在寻找一种将边界框添加到网页上显示的实时视频的有效方法。
该项目旨在使用内置的rtsp功能检测摄像机捕获的视频流（1080p 30fps）中的汽车，并在网页上显示结果。
我当前的解决方案：
使用OpenCV提取帧图像。我实时逐帧运行检测算法。首先将帧图像压缩为JPEG，然后编码为Base64字符串。然后，将Base64图像字符串以及边界框封装到JSON中。我使用WebSocket将JSON字符串传输到前端。
在前端，每次收到消息时，我都会使用画布绘制帧图像。
现在的问题是：
我想知道是否可以解决我的问题（达到30FPS），或者还有其他方法可以实现我的目的（同步后端检测结果和前端实时视频）< / strong>。
P.S。 边界框不会在后端渲染到图像，因为可能需要对前端的边界框执行一些操作，例如确定是否显示边界框。
编辑：后端的python中的代码（无检测）如下：
Main.py
from multiprocessing import Process, Queue
import base64
import json
import cv2
import WSServer
def capture(path, outq):
cap = cv2.VideoCapture(path)
while True:
ret, image = cap.read()
outq.put(image)
def conv2jpg(inq, outq):
while True:
image = inq.get()
imgEnc = cv2.imencode('.jpg', image, [cv2.IMWRITE_JPEG_QUALITY, 75])[1].tostring()
outq.put(imgEnc)
def conv2b64(inq, outq):
while True:
imgEnc = inq.get()
strEnc = str(base64.b64encode(imgEnc), 'utf-8')
outq.put(strEnc)
if __name__ == '__main__':
# After arg parsing
ws = WSServer.WSServer(args.ip, args.port)
capQ = Queue(2)
jpgQ = Queue(2)
b64Q = Queue(2)
capP = Process(target=capture, args=(args.input, capQ, ))
# args.input is the rtsp address of the camera
jpgP = Process(target=conv2jpg, args=(capQ, jpgQ, ))
b64P = Process(target=conv2b64, args=(jpgQ, b64Q, ))
b64P.start()
jpgP.start()
capP.start()
while True:
strEnc = b64Q.get()
msg = {'image': strEnc}
msgJson = json.dumps(msg)
# Send the json string over Websocket
USERS = ws.getUSERS()
for user in USERS:
user.put(msgJson)
WSServer.py
import threading
from queue import Queue
import asyncio
import websockets
class WSServer:
def __init__(self, ip="localhost", port="11000", interval=1.0/60):
self.ip = ip
self.port = port
self.interval = interval
self.USERS = set()
self.USERS_Lock = threading.Lock()
self.P = threading.Thread(target=self.proc)
self.P.setDaemon(True)
self.P.start()
async def get(self, q):
if q.empty():
return None
return q.get()
async def register(self, q):
with self.USERS_Lock:
self.USERS.add(q)
async def unregister(self, q):
with self.USERS_Lock:
self.USERS.remove(q)
async def serve(self, websocket, path):
q = Queue()
await self.register(q)
try:
while True:
while True:
msg = await self.get(q)
if type(msg) == type(None):
break
await websocket.send(msg)
await asyncio.sleep(self.interval)
finally:
await self.unregister(q)
def getUSERS(self):
with self.USERS_Lock:
return self.USERS.copy()
def proc(self):
print('Listening...')
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(websockets.serve(self.serve, self.ip, self.port))
loop.run_forever()
EDIT2：前端的简化代码：
<!DOCTYPE html>
<html>
<head>
<title>demo</title>
</head>
<body>
<img id="myImg" style="display: None;"></img>
<canvas id="myCanvas">
<script>
var img = document.getElementById('myImg')
var cvs = document.getElementById('myCanvas')
cvs.width = 1920
cvs.height = 1080
var ctx = cvs.getContext('2d')
var ws = new WebSocket("ws://127.0.0.1:11000/");
ws.onmessage = function (event) {
var msg = JSON.parse(event.data)
img.src = 'data:image/jpg;base64,' + msg.image
};
function render () {
ctx.drawImage(img, 0, 0)
window.requestAnimationFrame(render)
}
window.requestAnimationFrame(render)
</script>
</body>
</html>