Question

对于我目前的项目，我正在阅读iPhone的主摄像头输出。然后我通过以下方法将pixelbuffer转换为缓存的OpenGL纹理：CVOpenGLESTextureCacheCreateTextureFromImage。这在处理用于预览的相机帧时非常有用。使用iPhone 3GS，4，4S，iPod Touch（第4代）和IOS5，IOS6进行不同组合测试。

但是，对于具有非常高分辨率的实际最终图像，这仅适用于这些组合：

iPhone 3GS + IOS 5.1.1
iPhone 4 + IOS 5.1.1
iPhone 4S + IOS 6.0
iPod Touch（第4代）+ IOS 5.0

这不适用于：iPhone 4 + IOS6。

控制台中的确切错误消息：

Failed to create IOSurface image (texture)
2012-10-01 16:24:30.663 GLCameraRipple[676:907] Error at CVOpenGLESTextureCacheCreateTextureFromImage -6683

我通过改变Apple的GLCameraRipple项目来解决这个问题。您可以在此处查看我的版本：http://lab.bitshiftcop.com/iosurface.zip

以下是我将stilloutput添加到当前会话的方法：

- (void)setupAVCapture
{
    //-- Create CVOpenGLESTextureCacheRef for optimal CVImageBufferRef to GLES texture conversion.
    CVReturn err = CVOpenGLESTextureCacheCreate(kCFAllocatorDefault, NULL, [EAGLContext currentContext], NULL, &_videoTextureCache);
    if (err) 
    {
        NSLog(@"Error at CVOpenGLESTextureCacheCreate %d", err);
        return;
    }

    //-- Setup Capture Session.
    _session = [[AVCaptureSession alloc] init];
    [_session beginConfiguration];

    //-- Set preset session size.
    [_session setSessionPreset:_sessionPreset];

    //-- Creata a video device and input from that Device.  Add the input to the capture session.
    AVCaptureDevice * videoDevice = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
    if(videoDevice == nil)
        assert(0);

    //-- Add the device to the session.
    NSError *error;        
    AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:videoDevice error:&error];
    if(error)
        assert(0);

    [_session addInput:input];

    //-- Create the output for the capture session.
    AVCaptureVideoDataOutput * dataOutput = [[AVCaptureVideoDataOutput alloc] init];
    [dataOutput setAlwaysDiscardsLateVideoFrames:YES]; // Probably want to set this to NO when recording

    //-- Set to YUV420.
    [dataOutput setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA]
                                                             forKey:(id)kCVPixelBufferPixelFormatTypeKey]]; // Necessary for manual preview

    // Set dispatch to be on the main thread so OpenGL can do things with the data
    [dataOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()];


    // Add still output
    stillOutput = [[AVCaptureStillImageOutput alloc] init];
    [stillOutput setOutputSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey]];
    if([_session canAddOutput:stillOutput]) [_session addOutput:stillOutput];

    [_session addOutput:dataOutput];
    [_session commitConfiguration];

    [_session startRunning];
}

以下是我捕捉静止输出并处理它的方法：

- (void)capturePhoto
{
    AVCaptureConnection *videoConnection = nil;
    for (AVCaptureConnection *connection in stillOutput.connections) {
        for (AVCaptureInputPort *port in [connection inputPorts]) {
            if ([[port mediaType] isEqual:AVMediaTypeVideo] ) {
                videoConnection = connection;
                break;
            }
        }
        if (videoConnection) { break; }
    }

    [stillOutput captureStillImageAsynchronouslyFromConnection:videoConnection completionHandler:
     ^(CMSampleBufferRef imageSampleBuffer, NSError *error) {
         // Process hires image
         [self captureOutput:stillOutput didOutputSampleBuffer:imageSampleBuffer fromConnection:videoConnection];
     }];
}

以下是纹理的创建方式：

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
       fromConnection:(AVCaptureConnection *)connection
{
    CVReturn err;
    CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    size_t width = CVPixelBufferGetWidth(pixelBuffer);
    size_t height = CVPixelBufferGetHeight(pixelBuffer);

    if (!_videoTextureCache)
    {
        NSLog(@"No video texture cache");
        return;
    }

    if (_ripple == nil ||
        width != _textureWidth ||
        height != _textureHeight)
    {
        _textureWidth = width;
        _textureHeight = height;

        _ripple = [[RippleModel alloc] initWithScreenWidth:_screenWidth 
                                              screenHeight:_screenHeight
                                                meshFactor:_meshFactor
                                               touchRadius:5
                                              textureWidth:_textureWidth
                                             textureHeight:_textureHeight];

        [self setupBuffers];
    }

    [self cleanUpTextures];

    NSLog(@"%zi x %zi", _textureWidth, _textureHeight);

    // RGBA texture
    glActiveTexture(GL_TEXTURE0);
    err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault, 
                                                       _videoTextureCache,
                                                       pixelBuffer,
                                                       NULL,
                                                       GL_TEXTURE_2D,
                                                       GL_RGBA,
                                                       _textureWidth,
                                                       _textureHeight,
                                                       GL_BGRA,
                                                       GL_UNSIGNED_BYTE,
                                                       0,
                                                       &_chromaTexture);
    if (err) 
    {
        NSLog(@"Error at CVOpenGLESTextureCacheCreateTextureFromImage %d", err);
    }

    glBindTexture(CVOpenGLESTextureGetTarget(_chromaTexture), CVOpenGLESTextureGetName(_chromaTexture));
    glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); 
}

有关解决此问题的任何建议吗？

Answer 1

iPhone 4（以及iPhone 3GS和iPod Touch第4代）使用PowerVR SGX 535 GPU，maximum OpenGL ES texture size is 2048x2048。可以通过调用

找到此值

glGetIntegerv(GL_MAX_TEXTURE_SIZE, &maxTextureSize);

iPod Touch第四代。相机分辨率为720x960，iPhone 3GS为640x1136，但iPhone 4的后置摄像头分辨率为1936x2592，太大而不适合单一纹理。

您可以始终以较小的尺寸重写拍摄的图像，同时保留纵横比（1529x2048）。 Brad Larson在his GPUImage framework上做了这个，但它非常简单，只是使用Core Graphics重新绘制原始像素缓冲区的数据，然后从重绘数据中生成另一个像素缓冲区。框架的其余部分也是一个很好的资源。

Answer 2

我们无法将静态图像纹理投射到CVOpenGLESTextureCacheRef。 Core Video可让您将视频帧直接映射到OpenGL纹理。使用视频缓冲区，其中核心视频创建纹理并将它们提供给我们，已经在视频内存中。

要创建opengles纹理，此链接可能会对您有所帮助 Link

CVOpenGLESTextureCacheCreateTextureFromImage无法创建IOSurface

2 个答案: