Question

我一直试图在MonoTouch中进行一些实时视频图像处理。我正在使用AVCaptureSession从摄像机获取帧，该帧可以与AVCaptureVideoPreviewLayer一起使用。

我也成功地在我的委托类中获得了回调方法“DidOutputSampleBuffer”。但是，我尝试从生成的CMSampleBuffer创建UIImage的每种方式都失败了。

以下是设置捕获会话的代码：

captureSession = new AVCaptureSession ();
            captureSession.BeginConfiguration ();
            videoCamera = AVCaptureDevice.DefaultDeviceWithMediaType (AVMediaType.Video);

            if (videoCamera != null)
            {
                captureSession.SessionPreset = AVCaptureSession.Preset1280x720;

                videoInput = AVCaptureDeviceInput.FromDevice (videoCamera);

                if (videoInput != null)
                    captureSession.AddInput (videoInput);

                //DispatchQueue queue = new DispatchQueue ("videoFrameQueue");

                videoCapDelegate = new videoOutputDelegate (this);

                DispatchQueue queue = new DispatchQueue("videoFrameQueue");
                videoOutput = new AVCaptureVideoDataOutput ();

                videoOutput.SetSampleBufferDelegateAndQueue (videoCapDelegate, queue);
                videoOutput.AlwaysDiscardsLateVideoFrames = true;
                videoOutput.VideoSettings.PixelFormat = CVPixelFormatType.CV24RGB;

                captureSession.AddOutput (videoOutput);

                videoOutput.ConnectionFromMediaType(AVMediaType.Video).VideoOrientation = AVCaptureVideoOrientation.Portrait;

                previewLayer = AVCaptureVideoPreviewLayer.FromSession (captureSession);
                previewLayer.Frame = UIScreen.MainScreen.Bounds;
                previewLayer.AffineTransform = CGAffineTransform.MakeRotation (Convert.DegToRad (-90));
                //this.View.Layer.AddSublayer (previewLayer);

                captureSession.CommitConfiguration ();
                captureSession.StartRunning ();
            }

我尝试从样本缓冲区的图像缓冲区中生成的CVPixelBuffer创建一个CGBitmapContext，如下所示：

public override void DidOutputSampleBuffer (AVCaptureOutput captureOutput, MonoTouch.CoreMedia.CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
    {

        CVPixelBuffer pixelBuffer = sampleBuffer.GetImageBuffer () as CVPixelBuffer;
        CVReturn flag = pixelBuffer.Lock (0);
        if(flag == CVReturn.Success)
        {
            CGBitmapContext context = new CGBitmapContext
                    (
                        pixelBuffer.BaseAddress,
                        pixelBuffer.Width,
                        pixelBuffer.Height,
                        8,
                        pixelBuffer.BytesPerRow, 
                        CGColorSpace.CreateDeviceRGB (), 
                        CGImageAlphaInfo.PremultipliedFirst
                        );

            UIImage image = new UIImage(context.ToImage());

            ProcessImage (image);

            pixelBuffer.Unlock(0);

        }else
            Debug.Print(flag.ToString()

        sampleBuffer.Dispose();
    }

这会导致以下错误

<Error>: CGBitmapContextCreate: invalid data bytes/row: should be at least 2880 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst.

即使调整了一些参数，我也会得到一个无效的Handle异常或本机objective-c中的段错误。

我也试过简单地用CVImageBuffer创建一个CIImage并从中创建一个UIImage：

public override void DidOutputSampleBuffer (AVCaptureOutput captureOutput, MonoTouch.CoreMedia.CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
    {

        CIImage cImage = new CIImage(sampleBuffer.GetImageBuffer ());
        UIImage image = new UIImage(cImage);
        ProcessImage (image);

        sampleBuffer.Dispose();
    }

这在初始化CIImage时会导致异常：

NSInvalidArgumentException Reason: -[CIImage initWithCVImageBuffer:]: unrecognized selector sent to instance 0xc821d0

老实说，感觉就像是MonoTouch的某种错误，但是如果我遗漏了某些东西，或者只是想以一种奇怪的方式尝试这样做，请告诉我一些替代解决方案。

感谢

Answer 1

此错误消息说明了这一点：

<Error>: CGBitmapContextCreate: invalid data bytes/row: should be at least 2880 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst.

宽度为720像素，每行1084字节，每个像素超过1.5个字节 - 而不是RGB24（每个像素3个字节），它是一些平面格式。

您可能需要检查AVCaptureVideoDataOutput.AvailableVideoCVPixelFormatTypes是否有可用的像素格式，以查看是否有任何支持的格式更易于使用。

如何使用AVCaptureSession从CMSampleBuffer获取UIImage

1 个答案: