Question

我正在尝试修改来自AVFoundation的实时视频供稿的像素缓冲区，以通过OpenTok's API流式传输。但是，只要我尝试这样做并通过OpenTok的consumeFrame进行馈送，它就会崩溃。

我正在这样做，所以我可以应用不同的实时视频效果（滤镜，贴纸等）。我尝试使用不同的方法转换CGImage->CVPixelBuffer，但没有任何效果。

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    if !capturing || videoCaptureConsumer == nil {
        return
    }

    guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
        else {
            print("Error acquiring sample buffer")
            return
    }

    guard let videoInput = videoInput
        else {
            print("Capturer does not have a valid input")
            return
    }

    let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
    videoFrame.clearPlanes()

    videoFrame.timestamp = time
    let height = UInt32(CVPixelBufferGetHeight(imageBuffer))
    let width = UInt32(CVPixelBufferGetWidth(imageBuffer))

    if width != captureWidth || height != captureHeight {
        updateCaptureFormat(width: width, height: height)
    }

    // This is where I convert CVImageBuffer->CIImage, modify it, turn it into CGImage, then CGImage->CVPixelBuffer
    guard let finalImage = makeBigEyes(imageBuffer) else { return }

    CVPixelBufferLockBaseAddress(finalImage, CVPixelBufferLockFlags(rawValue: 0))

    videoFrame.format?.estimatedCaptureDelay = 10
    videoFrame.orientation = .left

    videoFrame.clearPlanes()
    videoFrame.planes?.addPointer(CVPixelBufferGetBaseAddress(finalImage))

    delegate?.finishPreparingFrame(videoFrame)

    videoCaptureConsumer!.consumeFrame(videoFrame)

    CVPixelBufferUnlockBaseAddress(finalImage, CVPixelBufferLockFlags(rawValue: 0))
}

这是我的CGImage->CVPixelBuffer方法：

    func buffer(from image: UIImage) -> CVPixelBuffer? {
    let attrs = [kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue] as CFDictionary
    var pixelBuffer : CVPixelBuffer?
    let status = CVPixelBufferCreate(kCFAllocatorDefault, Int(image.size.width), Int(image.size.height), kCVPixelFormatType_32ARGB, attrs, &pixelBuffer)
    guard (status == kCVReturnSuccess) else {
        return nil
    }

    CVPixelBufferLockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))
    let pixelData = CVPixelBufferGetBaseAddress(pixelBuffer!)

    let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
    let context = CGContext(data: pixelData, width: Int(image.size.width), height: Int(image.size.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer!), space: rgbColorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)

    context?.translateBy(x: 0, y: image.size.height)
    context?.scaleBy(x: 1.0, y: -1.0)

    UIGraphicsPushContext(context!)
    image.draw(in: CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height))
    UIGraphicsPopContext()
    CVPixelBufferUnlockBaseAddress(pixelBuffer!, CVPixelBufferLockFlags(rawValue: 0))

    return pixelBuffer
}

我在第一帧出现此错误：

*由于未捕获的异常'NSRangeException'而终止应用程序，原因：'* -[NSConcretePointerArray pointerAtIndex：]：尝试执行以下操作：超出边界1'的索引1处的访问指针

如果您到目前为止做的很不错，谢谢您的阅读。我已经在这个问题上停留了一段时间，因此任何指针都将不胜感激。谢谢。

Answer 1

由于要将摄像机（NV12）帧转换为RGB，因此需要在class MessageFrequencyType(DjangoObjectType): class Meta: model = MessageFrequency # Not sure about the name of your model class MessageFrequencyRangeType(DjangoObjectType): class Meta: model = MessageFrequencyRange # Not sure about the name of your model class Query(graphene.ObjectType): message_frequency = graphene.List(MessageFrequencyType) message_frequency_range = graphene.List(MessageFrequencyRangeType) def resolve_message_frequency(self, info, **kwargs): return MessageFrequency.objects.all() def resolve_message_frequency_range(self, info, **kwargs): return MessageFrequencyRange.objects.all() class CreateMessageFrequency(graphene.Mutation): chat_id = graphene.Int() date = graphene.Date() frequency = graphene.Int() class Arguments: chat_id = graphene.Int() date = graphene.Date() frequency = graphene.Int() def mutate(self, info, chat_id, date, frequency): message_frequency = MessageFrequency( chat_id=chat_id, date=date, frequency=frequency, ) message_frequency.save() return CreateMessageFrequency( chat_id=message_frequency.chat_id, date=message_frequency.date, frequency=message_frequency.frequency ) class CreateMessageFrequencyRange(graphene.Mutation): chat_id = graphene.Int() start_date = graphene.Date() end_date = graphene.Date() frequencies = graphene.Field(MessageFrequencyType) class Arguments: chat_id = graphene.Int() start_date = graphene.Date() end_date = graphene.Date() frequencies = graphene.Int() def mutate(self, info, chat_id, start_date, end_date, frequencies): chat_id=chat_id start_date=start_date end_date=end_date frequencies = MessageFrequency.objects.filter(id=frequencies).first() if not frequencies: raise Exception('Message Frequency not found') MessageFrequencyRange.objects.create( chat_id=chat_id, start_date=start_date, end_date=end_date, frequencies=frequencies ) return CreateMessageFrequencyRange( chat_id=chat_id, start_date=start_date, end_date=end_date, frequencies=frequencies ) class Mutation(graphene.ObjectType): create_message_frequency = CreateMessageFrequency.Field() create_message_frequency_range = CreateMessageFrequencyRange.Field()上将pixelFormat设置为OTPixelFormatARGB

TokBox：使用修改后的像素缓冲区时，“ consumeFrame”崩溃

1 个答案: