Question

我正在iPhone上捕捉实时视频帧并处理它们以使用OpenCV跟踪实时彩色斑点。为了最大限度地缩短处理时间，我发现帧纯处理时间取决于预设的FPS速率！我原本预计OpenCV模块中图像的处理时间只取决于图像大小和OpenCV算法本身，不是吗？

setupCaptureSession：

    try {
        double cdebts = 0;

        String rq3 = "select credit_balance from credit where user_id = '" + uid + "'";
        st3 = conn.createStatement();
        ResultSet rs3 = st3.executeQuery(rq3);

        while ((rs3.next())) {
            cdebts = rs3.getDouble(1);
            totalcdebts += cdebts;
        }

    } // end try
    catch (Exception i) {
        System.out.print(i);
    } // end catch

captureOutput：

- (void)setupCaptureSession
{
  if ( _captureSession )
  {
     return;
  }
  _captureSession = [[AVCaptureSession alloc] init];
  videoPreviewLayer = [AVCaptureVideoPreviewLayer layerWithSession:_captureSession];
  AVCaptureDevice *videoDevice = [self frontCamera];
  NSError *videoDeviceError = nil;
  AVCaptureDeviceInput *videoIn = [[AVCaptureDeviceInput alloc] initWithDevice:videoDevice error:&videoDeviceError];
  if ( [_captureSession canAddInput:videoIn] )
  {
    [_captureSession addInput:videoIn];
    _videoDevice = videoDevice;
  }
  else
  {
    [self handleNonRecoverableCaptureSessionRuntimeError:videoDeviceError];
    return;
  }
  AVCaptureVideoDataOutput *videoOut = [[AVCaptureVideoDataOutput alloc] init];
  videoOut.videoSettings = [NSDictionary dictionaryWithObject:
                          [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey];
  [videoOut setSampleBufferDelegate:self queue:_videoDataOutputQueue];
  videoOut.alwaysDiscardsLateVideoFrames = NO;
  if ( [_captureSession canAddOutput:videoOut] )
  {
     [_captureSession addOutput:videoOut];
  }
  _videoConnection = [videoOut connectionWithMediaType:AVMediaTypeVideo];
  _videoBufferOrientation = _videoConnection.videoOrientation;
  [self configureCameraForFrameRate :  videoDevice];
  return;
}

在processImageWithOpenCV的开头和结尾分别放置：

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection (AVCaptureConnection *)connection
  {
    UIImage *sourceUIImage = [self imageFromSampleBuffer : sampleBuffer];
    CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription( sampleBuffer );
    if ( self.outputVideoFormatDescription == NULL )
    {
       [self setupVideoPipelineWithInputFormatDescription:formatDescription];
    }
    else
    {
       processedFrameNumber++;
       @synchronized( _renderer )
       {
           [_renderer processImageWithOpenCV : sourceUIImage : processingData];
       }
    }
}

和

timeBeforeProcess = [NSDate timeIntervalSinceReferenceDate];

对于FPS值20,30,40和60，我分别测量了以下帧处理时间值，单位为ms 0.0140,0.0089,0.0074和0.0072。

为了说明，此图表显示帧处理时间如何随FPS减小：

graph

你有任何解释吗？

谢谢。

Answer 1

iOS可以扩展资源以节省电池/减少热量吗？较低的频率可能足够用较少的cpu / gpu / ...，但不是较高的频率。许多手臂供应商都采用“大+小”架构。

可能有一个log / perf的东西你可以查询，但如果没有，你可以尝试更新的和更旧的iphone吗？我怀疑iphone 8或9比5或6更具侵略性。请注意，修复突然放电可能会影响您在这些设备上的实验。

为什么在iPhone上处理视频帧的时间取决于FPS速率？

1 个答案: