Question

我一直在从事一个使用树莓派将实时视频供稿发送到服务器的项目。这有点奏效，但我不希望如此。问题主要是速度。现在，我可以发送640x480的视频流，其速度约为3.5 FPS，而发送1920x1080的图像，约为0.5 FPS，这太糟糕了。由于我不是专业人员，所以我认为应该有一种方法来改进我的代码。

发件人（Raspberry pi）：

def send_stream():
    connection = True
    while connection:
        ret,frame = cap.read()
        if ret:
            # You might want to enable this while testing.
            # cv2.imshow('camera', frame)
            b_frame = pickle.dumps(frame)
            b_size = len(b_frame)
            try:
                s.sendall(struct.pack("<L", b_size) + b_frame)
            except socket.error:
                print("Socket Error!")
                connection = False

        else:
            print("Received no frame from camera, exiting.")
            exit()

接收方（服务器）：

    def recv_stream(self):
        payload_size = struct.calcsize("<L")
        data = b''
        while True:
            try:
                start_time = datetime.datetime.now()
                # keep receiving data until it gets the size of the msg.
                while len(data) < payload_size:
                    data += self.connection.recv(4096)
                # Get the frame size and remove it from the data.
                frame_size = struct.unpack("<L", data[:payload_size])[0]
                data = data[payload_size:]
                # Keep receiving data until the frame size is reached.
                while len(data) < frame_size:
                    data += self.connection.recv(32768)
                # Cut the frame to the beginning of the next frame.
                frame_data = data[:frame_size]
                data = data[frame_size:]

                frame = pickle.loads(frame_data)
                frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

                end_time = datetime.datetime.now()
                fps = 1/(end_time-start_time).total_seconds()
                print("Fps: ",round(fps,2))

                self.detect_motion(frame,fps)

                self.current_frame = frame

            except (socket.error,socket.timeout) as e:
                # The timeout got reached or the client disconnected. Clean up the mess.
                print("Cleaning up: ",e)
                try:
                    self.connection.close()
                except socket.error:
                    pass
                self.is_connected = False
                break

Answer 1

一个潜在的原因可能是由于读取帧时的I / O延迟。由于cv2.VideoCapture().read()是阻塞操作，因此主程序将停顿，直到从相机设备读取一帧并返回为止。一种提高性能的方法是生成另一个线程以 parallel 的方式处理抓帧，而不是依赖单个线程以 sequential 的顺序抓帧。我们可以通过创建一个仅轮询新帧而主线程处理/绘制最新帧的新线程来提高性能。

您当前的方法（顺序）：

线程1：抓取框架->处理框架->绘制

建议的方法（并行）：

线程1：抢帧

from threading import Thread
import time

def get_frames():
    while True:
        ret, frame = cap.read()
        time.sleep(.01)

thread_frames = Thread(target=self.get_frames, args=())
thread_frames.daemon = True
thread_frames.start()

线程2：处理框架->绘图

def process_frames():
    while True:
        # Grab most recent frame
        # Process/plot frame
        ...

由于具有单独的线程，因此您的程序将是并行的，因为始终会有准备好要处理的帧，而不必等待帧被读入才可以进行处理。

注意：：此方法将基于I / O延迟减少而提高性能。这并不是FPS的真正提高，因为这是延迟的大幅降低（一帧始终可用于处理；我们无需轮询摄像头设备并等待I / O去完成）。

Answer 2

在互联网上搜索了很长时间后，我找到了一种快速解决方案，使fps翻了一番（这仍然太低了：@ 1080p时为1.1 fps）。我所做的是我停止使用pickle并改为使用base64。显然，腌制图像仅需要一段时间。无论如何，这是我的新代码：

发件人（Raspberry pi）：

def send_stream():
global connected
connection = True
while connection:
    if last_frame is not None:

        # You might want to uncomment these lines while testing.
        # cv2.imshow('camera', frame)
        # cv2.waitKey(1)
        frame = last_frame

        # The old pickling method.
        #b_frame = pickle.dumps(frame)

        encoded, buffer = cv2.imencode('.jpg', frame)
        b_frame = base64.b64encode(buffer)

        b_size = len(b_frame)
        print("Frame size = ",b_size)
        try:
            s.sendall(struct.pack("<L", b_size) + b_frame)
        except socket.error:
            print("Socket Error!")
            connection = False
            connected = False
            s.close()
            return "Socket Error"
    else:
        return "Received no frame from camera"

接收方（服务器）：

    def recv_stream(self):
    payload_size = struct.calcsize("<L")
    data = b''
    while True:
        try:
            start_time = datetime.datetime.now()
            # keep receiving data until it gets the size of the msg.
            while len(data) < payload_size:
                data += self.connection.recv(4096)
            # Get the frame size and remove it from the data.
            frame_size = struct.unpack("<L", data[:payload_size])[0]
            data = data[payload_size:]
            # Keep receiving data until the frame size is reached.
            while len(data) < frame_size:
                data += self.connection.recv(131072)
            # Cut the frame to the beginning of the next frame.
            frame_data = data[:frame_size]
            data = data[frame_size:]

            # using the old pickling method.
            # frame = pickle.loads(frame_data)

            # Converting the image to be sent.
            img = base64.b64decode(frame_data)
            npimg = np.fromstring(img, dtype=np.uint8)
            frame = cv2.imdecode(npimg, 1)

            frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

            end_time = datetime.datetime.now()
            fps = 1/(end_time-start_time).total_seconds()
            print("Fps: ",round(fps,2))
            self.detect_motion(frame,fps)

            self.current_frame = frame

        except (socket.error,socket.timeout) as e:
            # The timeout got reached or the client disconnected. Clean up the mess.
            print("Cleaning up: ",e)
            try:
                self.connection.close()
            except socket.error:
                pass
            self.is_connected = False
            break

我还增加了数据包大小，从而在测试时从本地计算机发送到本地计算机时提高了fps，但这在使用树莓派时并没有任何改变。

您可以在我的github上看到完整的代码：https://github.com/Ruud14/SecurityCamera

如何改善python openCV视频流？

2 个答案: