在Mac OS X 10.10.2上使用waitKey的opencv imshow太慢了（使用c ++）

Question

我在Mac OS X 10.10.2上使用opencv c ++来处理视频帧并显示它们。用waitKey显示视频的imshow的性能非常慢。

我有以下代码正确显示HD（1920x1080）灰度帧，但它运行速度大约10倍太慢（即每秒2到3帧而不是每秒30帧）。

          cv::Mat framebuf[TEST_COUNT];

    //--- Code here to allocate and fill the frame buffer with about 4 seconds of video. This part works correctly.

        //--- This loop runs too slow by factor of approximately 10x
    for (int f = 0; f < TEST_COUNT; f++)
    {
        cv::imshow(windowName, framebuf[f]);
        cv::waitKey(33);
    }

有人可以建议如何从opencv imshow（）获得实时或接近实时的性能吗？我看过很多帖子说它们实时显示视频甚至比实时更快，所以我不确定我做错了什么。任何帮助将不胜感激。

Answer 1

我可能错了，但对我来说，问题不在于您的代码，而在于您的操作系统/配置。我写过一个小测试：

import cv2
import numpy as np
from random import randrange
img = np.zeros((1920, 1080), dtype = np.uint8)
counter = 0
while counter < 1000:
    cv2.line(img, (randrange(0, 1920), randrange(0, 1080)), (randrange(0, 1920), randrange(0, 1080)), (randrange(0, 255)))
    cv2.imshow('test', img)
    temp = cv2.waitKey(1)
    counter += 1
    print counter

在我的机器上（Core 2 duo 2,6Ghz x64,8gb ram，ssd）完成此测试需要大约30秒。运行它，如果你得到比definitelly更大的时间你的笔记本电脑/ opencv配置错误。我在Mac OS X上使用过OpenCV 2.4.x（我觉得它是10.9）并且运行正常。重新安装OpenCV是我脑海中最明显的解决方案。当你删除OpenCV时，使用brew再次安装它 - brew install opencv --with-tbb --with-python --with-ffpmeg（或类似的东西 - 使用brew options opencv检查）应该没问题。第一个选项告诉brew使用tbb构建opencv（线程构建块 - 用于多线程的库，有时可以显着提高速度），第二个用于安装python包装器，最后一个用于安装ffmpeg（处理编解码器等）。

Answer 2

您必须减少功能等待键的输入。尝试使用2-5范围内的较小数字。它还取决于您同时运行的其他进程，尝试关闭其他进程并查看是否有所改进

Answer 3

您可以创建自己的窗口以显示图像。将MyWindow.m MyWindow.h文件添加到项目中。

MyWindow.h

#ifndef MY_WINDOW_H
#define MY_WINDOW_H

#ifdef __cplusplus
extern "C" {
#endif
    void* createNSWindow(int x, int y, int w, int h);
    void renderNSWindow(void* inwindow, void* data, int w, int h, int c);
    void processNSEvent();
#ifdef __cplusplus
}
#endif

#endif

在main.cpp中的用法不要忘了waitKey

#include "MyWindow.h"

// need create a cv window and do nothing
cv::namedWindow("xxx", 1);

// create window
void* w = createNSWindow(0, 0, 0, 0);

// frame image
cv::Mat frameImage;

// render loop
renderNSWindow(w, frameImage.data, frameImage.cols, frameImage.rows, frameImage.channels());

// need waitKey to display window
processNSEvent();

实现，在MyWindow.m中，删除导入“ MyWindow.h”

#import <Cocoa/Cocoa.h>

@interface MyWindow : NSWindow
@property(nonatomic, strong) NSImageView *imgv;
@end

@implementation MyWindow
@end


static NSImage* _createNSImage(void* data, int w, int h, int c);

void* createNSWindow(int x, int y, int w, int h) {

    NSRect screenFrame = [[NSScreen mainScreen] frame];
    NSRect frame = NSMakeRect(x, y, w, h);
    if (w == 0 || h == 0) {
        frame = screenFrame;
    }

    MyWindow* window  = [[MyWindow alloc] initWithContentRect:frame
                                                    styleMask:NSWindowStyleMaskBorderless
                                                      backing:NSBackingStoreBuffered
                                                        defer:NO] ;


    //_initApp(window);

    [window makeKeyAndOrderFront:NSApp];
    window.titleVisibility = TRUE;
    window.styleMask = NSWindowStyleMaskResizable | NSWindowStyleMaskTitled |NSWindowStyleMaskFullSizeContentView;

    window.imgv = [[NSImageView alloc] initWithFrame:NSMakeRect(0, 0, frame.size.width, frame.size.height)];
    [window.contentView addSubview:window.imgv];


    return (void*)CFBridgingRetain(window);
}


static NSImage* _createNSImage(void* data, int w, int h, int c) {

    size_t bufferLength = w * h * c;

    CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, data, bufferLength, NULL);
    size_t bitsPerComponent = 8;
    size_t bitsPerPixel = c * bitsPerComponent;
    size_t bytesPerRow = c * w;
    CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
    CGBitmapInfo bitmapInfo = kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedLast;
    if (c < 4) {
        bitmapInfo = kCGBitmapByteOrderDefault | kCGImageAlphaNone;
        unsigned char* buf = data;
        for(int i = 0; i < w*h; i++) {
            unsigned char temp = buf[i*c];
            buf[i*c] = buf[i*c+c-1];
            buf[i*c+c-1] = temp;
        }
    }
    CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;

    CGImageRef iref = CGImageCreate(w,
                                    h,
                                    bitsPerComponent,
                                    bitsPerPixel,
                                    bytesPerRow,
                                    colorSpaceRef,
                                    bitmapInfo,
                                    provider,   // data provider
                                    NULL,       // decode
                                    YES,        // should interpolate
                                    renderingIntent);

    NSImage* image = [[NSImage alloc] initWithCGImage:iref size:NSMakeSize(w, h)];
    return image;
}

void renderNSWindow(void* inwindow, void* data, int w, int h, int c) {
    MyWindow* window = (__bridge MyWindow*)inwindow;

    window.imgv.image = _createNSImage(data, w, h, c);

}

void processNSEvent() {
    for (;;)
    {
        NSEvent* event = [NSApp nextEventMatchingMask:NSEventMaskAny
                                            untilDate:[NSDate distantPast]
                                               inMode:NSDefaultRunLoopMode
                                              dequeue:YES];
        if (event == nil)
            break;

        [NSApp sendEvent:event];
    }
}

其他事情，waitKey现在大约需要20毫秒，您可以在后台线程中执行OpenCV，并在主线程中显示窗口。还可以使用processNSEvent代替仅耗时约10ms的waitKey。

完整的源代码：

#include <iostream>
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <dispatch/dispatch.h>
#include "MyWindow.h"


using namespace std;
using namespace cv;


int opencvfunc(int argc, const char *argv[]);
bool newFrame = false;
cv::Mat back_frame;

int main(int argc, const char * argv[]) {
    cv::namedWindow("render", 1);

    void* w = createNSWindow(0, 0, 0, 0);

    dispatch_queue_t opencvq = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0);
    dispatch_async(opencvq, ^{
        opencvfunc(argc, argv);
    });


    while(true) {
        usleep(3*1000);
        if(newFrame) {
            std::chrono::system_clock::time_point starttime = std::chrono::system_clock::now();
            renderNSWindow(w, back_frame.data, back_frame.cols, back_frame.rows, back_frame.channels());
            newFrame = false;
            //auto key = cv::waitKey(1);
            //if (key == 'q') {
            //    break;
            //}
            processNSEvent();
            std::chrono::system_clock::time_point endtime = std::chrono::system_clock::now();
            std::cout << "imshow:" << std::chrono::duration_cast<std::chrono::duration<double>>(endtime-starttime).count()*1000 << std::endl;
        }
    }

    return 0;
}

int opencvfunc(int argc, const char *argv[]) {

    cv::VideoCapture cap;
    cap.open(0);
    if (!cap.isOpened()) {
        std::cout << "Couldn't open camera 0." << endl;
        return EXIT_FAILURE;
    }

    Mat frame, unmodified_frame;

    for (;;) {

        cap >> frame; // get a new frame from camera
        if (frame.empty()) { // stop if we're at the end of the video
            break;
        }


        //unmodified_frame = frame.clone();

        // ...

        back_frame = frame.clone();
        newFrame = true;
    }

    return EXIT_SUCCESS;
}

Answer 4

OpenCV 4已解决此问题，请更新到新版本。

还有一件事情，处理视频并在两个线程中显示视频。

#include <stdio.h>
#include <iostream>

#include <opencv2/opencv.hpp>
#include <dispatch/dispatch.h>

using namespace cv;
using namespace std;


bool newFrame = false;
Mat back_frame;

int opencvmain(int argc, char** argv ) {

    // open camear
    cv::VideoCapture cap;
    cap.open(0);
    if (!cap.isOpened()) {
        std::cout << "Couldn't open camera 0." << std::endl;
        return EXIT_FAILURE;
    }

    // define frame images
    cv::Mat frame;

    // frame loop
    for (;;) {

        // get video frame
        cap >> frame;
        if (frame.empty()) {
            break;
        }

        // render
        back_frame = frame.clone();
        newFrame = true;        
    }
    return 0;
}

int main(int argc, char** argv ) {


    namedWindow("video", WINDOW_AUTOSIZE );

    dispatch_queue_t opencvq = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
    dispatch_async(opencvq, ^{
        opencvmain(argc, argv);
    });


    while(true) {
        usleep(3*1000);
        if(newFrame) {                
            imshow("video", back_frame);
            auto key = cv::waitKey(1);
            if (key == ' ') {
                break;
            }
            newFrame = false;                
        }
    }

    return 0;
}