Question

目前，使用iPhone 5s / 6，我可以将120张（iPhone 5s）或240张（iPhone 6）帧/秒捕获到CMSampleBufferRef中。但是，返回给我的AVCaptureDeviceFormat仅提供这些高速帧速率，分辨率为1280x720。

我想以更低的分辨率（640x480或更低）捕获它，因为我将它放入循环缓冲区用于存储目的。虽然我能够降低didOutputSampleBuffer委托方法的分辨率，但我想知道CMSampleBufferRef是否有办法通过配置设备或设置直接为我提供较低的分辨率，而不是采用720p图像并降低使用CVPixelBuffer手动分辨率。

我需要将图像存储在缓冲区中以便以后处理，并希望应用必要的最小处理，否则我将开始删除帧。如果我可以避免调整大小并直接从didOutputSampleBuffer委托方法获得较低分辨率的CMSampleBuffer，那将是理想的。

在240fps时，我需要在5ms内处理每个图像，并且调整大小例程无法跟上以此速率缩小图像的速度。但是，我想将它存储到循环缓冲区中以便以后处理（例如使用AVAssetWriter写出电影）但需要较低的分辨率。

似乎高帧率录制支持的唯一图像尺寸为1280x720。将此分辨率的多个图像放入帧缓冲区将产生内存压力，因此我希望直接从didOutputSampleBuffer捕获较低分辨率的图像，如果它可以节省内存并保持帧速率。

感谢您的协助。

Answer 1

// core image use GPU to all image ops, crop / transform / ...

// --- create once ---
EAGLContext *glCtx = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
CIContext *ciContext = [CIContext contextWithEAGLContext:glCtx options:@{kCIContextWorkingColorSpace:[NSNull null]}];
// use rgb faster 3x
CGColorSpaceRef ciContextColorSpace = CGColorSpaceCreateDeviceRGB();
OSType cvPixelFormat = kCVPixelFormatType_32BGRA;

// create compression session
VTCompressionSessionRef compressionSession;
NSDictionary* pixelBufferOptions = @{(__bridge NSString*) kCVPixelBufferPixelFormatTypeKey:@(cvPixelFormat),
                                     (__bridge NSString*) kCVPixelBufferWidthKey:@(outputResolution.width),
                                     (__bridge NSString*) kCVPixelBufferHeightKey:@(outputResolution.height),
                                     (__bridge NSString*) kCVPixelBufferOpenGLESCompatibilityKey : @YES,
                                     (__bridge NSString*) kCVPixelBufferIOSurfacePropertiesKey : @{}};

OSStatus ret = VTCompressionSessionCreate(kCFAllocatorDefault,
                                          outputResolution.width,
                                          outputResolution.height,
                                          kCMVideoCodecType_H264,
                                          NULL,
                                          (__bridge CFDictionaryRef)pixelBufferOptions,
                                          NULL,
                                          VTEncoderOutputCallback,
                                          (__bridge void*)self,
                                          &compressionSession);

CVPixelBufferRef finishPixelBuffer;
// I'm use VTCompressionSession pool, you can use AVAssetWriterInputPixelBufferAdaptor
CVReturn res = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, VTCompressionSessionGetPixelBufferPool(compressionSession), &finishPixelBuffer);
// -------------------

// ------ scale ------
// new buffer comming...
// - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection

CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);

CIImage *baseImg = [CIImage imageWithCVPixelBuffer:pixelBuffer];
CGFloat outHeight = 240;
CGFloat scale = 1 / (CVPixelBufferGetHeight(pixelBuffer) / outHeight);
CGAffineTransform transform = CGAffineTransformMakeScale(scale, scale);

// result image not changed after
CIImage *resultImg = [baseImg imageByApplyingTransform:transform];
// resultImg = [resultImg imageByCroppingToRect:...];

// CIContext applies transform to CIImage and draws to finish buffer
[ciContext render:resultImg toCVPixelBuffer:finishPixelBuffer bounds:resultImg.extent colorSpace:ciContextColorSpace];
CVPixelBufferUnlockBaseAddress(pixelBuffer, kCVPixelBufferLock_ReadOnly);

// [videoInput appendSampleBuffer:CMSampleBufferCreateForImageBuffer(... finishPixelBuffer...)]
VTCompressionSessionEncodeFrame(compressionSession, finishPixelBuffer, CMSampleBufferGetPresentationTimeStamp(sampleBuffer), CMSampleBufferGetDuration(sampleBuffer), NULL, sampleBuffer, NULL);
// -------------------

使用低分辨率将AVCaptureVideoDataOutput捕获120/240 fps到帧缓冲区

1 个答案: