Question

我有一个UIImage，它是以前从png文件创建的：

let strokeUIImage = UIImage(data: pngData)

我想将strokeImage（具有不透明性）转换为MTLTexture，以便在MTKView中显示，但是执行转换似乎执行了不必要的预乘，从而使所有半透明的边缘变暗。

我的混合设置如下：

pipelineDescriptor.colorAttachments[0].isBlendingEnabled = true
pipelineDescriptor.colorAttachments[0].rgbBlendOperation = .add
pipelineDescriptor.colorAttachments[0].alphaBlendOperation = .add
pipelineDescriptor.colorAttachments[0].sourceRGBBlendFactor = .one
pipelineDescriptor.colorAttachments[0].sourceAlphaBlendFactor = .one
pipelineDescriptor.colorAttachments[0].destinationRGBBlendFactor = .oneMinusSourceAlpha
pipelineDescriptor.colorAttachments[0].destinationAlphaBlendFactor = .oneMinusSourceAlpha

我尝试了两种转换方法：

let stampTexture = try! MTKTextureLoader(device: self.device!).newTexture(cgImage: strokeUIImage.cgImage!, options: nil)

以及更精细的dataProvider驱动方法：

let image = strokeUIImage.cgImage!
    let imageWidth = image.width
    let imageHeight = image.height
    let bytesPerPixel:Int! = 4
    let rowBytes = imageWidth * bytesPerPixel        

    let texDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: .rgba8Unorm_srgb,
                                                              width: imageWidth,
                                                              height: imageHeight,
                                                              mipmapped: false)

    guard let stampTexture = device!.makeTexture(descriptor: texDescriptor) else { return }

    let srcData: CFData! = image.dataProvider?.data
    let pixelData = CFDataGetBytePtr(srcData)

    let region = MTLRegionMake2D(0, 0, imageWidth, imageHeight)
    stampTexture.replace(region: region, mipmapLevel: 0, withBytes: pixelData!, bytesPerRow: Int(rowBytes))

两者都会产生相同的不需要的预乘结果。

我尝试了后者，因为有一些帖子表明旧的swift3方法CGDataProviderCopyData()从未预乘的图像中提取原始像素数据。可悲的是，等价物：

let srcData: CFData! = image.dataProvider?.data

似乎并不能解决问题。我想念什么吗？

任何指针将不胜感激。

Answer 1

经过大量实验，我得出了一个解决方案，该解决方案解决了CoreGraphics图像固有的预乘问题。由于沃伦（Warren）关于使用Accelerate函数（特别是vImageUnpremultiplyData_ARGB8888）的提示，我想，为什么不使用vImage_CGImageFormat构建CGImage，这将允许我使用指定如何解释alpha的bitmapInfo设置。 ..结果不完美，如以下图片附件所示：

不知何故，在翻译中，alpha值会略微打乱（可能是rgb，但也不是很大）。顺便说一句，我应该指出png像素格式是sRGB，并且我正在使用的MTKView设置为MTLPixelFormat.rgba16Float（应用要求）

下面是我实现的完整metalDrawStrokeUIImage例程。这行特别值得注意：

bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue)

基本上取消了Alpha的关联（我认为）而没有调用vImageUnpremultiplyData_ARGB8888。查看生成的图像肯定看起来像是未预乘的图像...

最后，要获得MTKView端的预乘纹理，我让片段着色器处理预乘：

fragment float4 premult_fragment(VertexOut interpolated [[stage_in]],
                                 texture2d<float> texture [[texture(0)]],
                                 sampler sampler2D [[sampler(0)]]) {
  float4 sampled = texture.sample(sampler2D, interpolated.texCoord);

  // this fragment shader premultiplies incoming rgb with texture's alpha

  return float4(sampled.r * sampled.a,
                sampled.g * sampled.a,
                sampled.b * sampled.a,
                sampled.a );


} // end of premult_fragment

结果非常接近输入源，但是图像的不透明性可能比传入的png高5％。再次，png像素格式是sRGB，并且我要用来显示的MTKView设置为MTLPixelFormat.rgba16Float。因此，我敢肯定某个地方会变暗。如果有人有任何指点，我将不胜感激。

下面是相关代码的其余部分：

func metalDrawStrokeUIImage (strokeUIImage: UIImage, strokeBbox: CGRect) {

self.metalSetupRenderPipeline(compStyle: compMode.strokeCopy) // needed so stampTexture is not modified by fragmentFunction

let bytesPerPixel = 4
let bitsPerComponent = 8

let width = Int(strokeUIImage.size.width)
let height = Int(strokeUIImage.size.height)

let rowBytes = width * bytesPerPixel
//
let texDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: .rgba8Unorm_srgb,
                                                             width: width,
                                                             height: height,
                                                             mipmapped: false)
guard let stampTexture = device!.makeTexture(descriptor: texDescriptor) else { return }

//let cgImage: CGImage = strokeUIImage.cgImage!
//let sourceColorSpace = cgImage.colorSpace else {
guard
  let cgImage = strokeUIImage.cgImage,
  let sourceColorSpace = cgImage.colorSpace else {
    print("Unable to initialize cgImage or colorSpace.")
    return
}

var format = vImage_CGImageFormat(
  bitsPerComponent: UInt32(cgImage.bitsPerComponent),
  bitsPerPixel: UInt32(cgImage.bitsPerPixel),
  colorSpace: Unmanaged.passRetained(sourceColorSpace),
  bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue),
  version: 0, decode: nil,
  renderingIntent: CGColorRenderingIntent.defaultIntent)
var sourceBuffer = vImage_Buffer()

defer {
  free(sourceBuffer.data)
}

var error = vImageBuffer_InitWithCGImage(&sourceBuffer, &format, nil, cgImage, numericCast(kvImageNoFlags))

guard error == kvImageNoError else {
  print ("[MetalBrushStrokeView]: can't vImageBuffer_InitWithCGImage")
  return

}

//vImagePremultiplyData_RGBA8888(&sourceBuffer, &sourceBuffer, numericCast(kvImageNoFlags))


// create a CGImage from vImage_Buffer
var destCGImage = vImageCreateCGImageFromBuffer(&sourceBuffer, &format, nil, nil, numericCast(kvImageNoFlags), &error)?.takeRetainedValue()


guard error == kvImageNoError else {
  print ("[MetalBrushStrokeView]: can't vImageCreateCGImageFromBuffer")
  return
}

let dstData: CFData = (destCGImage!.dataProvider!.data)!
let pixelData = CFDataGetBytePtr(dstData)

destCGImage = nil

let region = MTLRegionMake2D(0, 0, Int(width), Int(height))
stampTexture.replace(region: region, mipmapLevel: 0, withBytes: pixelData!, bytesPerRow: Int(rowBytes))
let stampColor = UIColor.white
let stampCorners = self.stampSetVerticesFromBbox(bbox: strokeBbox)
self.stampAppendToVertexBuffer(stampLayer: stampLayerMode.stampLayerFG, stampCorners: stampCorners, stampColor: stampColor)
self.metalRenderStampSingle(stampTexture: stampTexture)
self.initializeStampArray() // clears out the stamp array so we always draw 1 stamp at a time


} // end of func metalDrawStrokeUIImage (strokeUIImage: UIImage, strokeBbox: CGRect)

无需预先乘法即可将CGImage转换为MTLTexture

1 个答案: