Question

我一直在探索CoreML和实时图像分类，偶然发现了巨大的性能提升，将示例缓冲区转换为我无法解释的UIImage。

我发现CoreML加剧了这种情况，但没有它仍然存在。

背景：我正在使用AVCaptureVideoDataOutput.setSampleBufferDelegate通过captureOutput回调将相机帧解码为UIImage。

简而言之，为什么使用这种方法

 private func slowImageFromSampleBuffer(sampleBuffer: CMSampleBuffer) -> UIImage? {
    // Why is this approach so much slower?
    // Approximately 4x slower (8ms as opposed to 2ms) and jumps up to 100ms when in contention
    // with CoreML image classification.
    guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return nil }
    let ciImage = CIImage(cvPixelBuffer: imageBuffer)
    guard let cgImage = context.createCGImage(ciImage, from: ciImage.extent) else { return nil }
    return UIImage(cgImage: cgImage)
}

比这慢四倍

 private func fasterImageFromSampleBuffer(sampleBuffer: CMSampleBuffer) ->UIImage? {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
        return nil
    }
    CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly)
    defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly) }

    let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)
    let width = CVPixelBufferGetWidth(pixelBuffer)
    let height = CVPixelBufferGetHeight(pixelBuffer)
    let bytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let bitmapInfo:CGBitmapInfo = [.byteOrder32Little, CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)]

    guard let context = CGContext(data: baseAddress, width: width, height: height, bitsPerComponent: 8, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue) else {
        return nil
    }
    guard let cgImage = context.makeImage() else { return nil }
    let image = UIImage(cgImage: cgImage)
    return image
}

人们会认为context.createCGImage会像手动操作CGContext一样出色。 ~~我目前的理论是，它默认使用与CoreML竞争的GPU，但这只是个猜测。~~我关闭了CoreML，但性能至少有2倍的差异。

感谢您的见解。其他信息可在以下要点中找到：

https://gist.github.com/dimroc/7d28c247b85f77e5b942d8927b04fb90

为什么这两种imageFromSample转换方法之间存在如此速度差异？

0 个答案: