CoreGraphics - 无法用UnsafeMutablePointer <uint8>

时间:2017-05-21 02:33:43

标签: ios swift core-graphics

我刚刚开始搞乱图像处理,我有几个非常奇怪的问题,或者至少我认为它们是。我假设我犯了一些非常愚蠢的错误。

我打算发布另一个关于此的问题,但是,有时我会得到随机噪声而不是用户绘制数字的像素表示。如果有人能告诉我为什么会发生这种情况,我将不胜感激。我很难找到原因,因为我读到的所有内容都表明这段代码应该有用。

如果有人需要更多信息,请告诉我们!感谢您的帮助!

目标:
首先,获取用户在屏幕上绘制的数字。然后,将图像的大小调整为28 x 28.接下来,将图像转换为灰度,并获得标准化像素值的数组。最后,将标准化的灰度像素值输入到机器学习算法中。

[注意:在下面的图片中,点代表0值,1代表值&gt; 0。]

以下代码的输出效果很好。如果用户绘制“3”,我通常会得到如下内容:

UInt32

问题:

如果我将UnsafeMutablePointer和Buffer的大小更改为UInt8,我会看到随机噪音。或者,如果我用[UInt32](repeating: 0, count: totalBytes)甚至[UInt8](repeating: 0, count: totalBytes)替换UnsafeMutablePointer和Buffer,则每个像素最终为0,这是我真的不明白。

如果我将UnsafeMutablePointer和Buffer的大小更改为UInt8,则以下是像素的输出:

UInt8

获取灰度像素的代码:

public extension UIImage
{
    private func grayScalePixels() -> UnsafeMutableBufferPointer<UInt32>?
    {
        guard let cgImage = self.cgImage else { return nil }

        let bitsPerComponent = 8
        let width = cgImage.width
        let height = cgImage.height
        let totalBytes = (width * height)
        let colorSpace = CGColorSpaceCreateDeviceGray()
        let data = UnsafeMutablePointer<UInt32>.allocate(capacity: totalBytes)
        defer { data.deallocate(capacity: totalBytes) }

        guard let imageContext = CGContext(data: data, width: width, height: height, bitsPerComponent: bitsPerComponent, bytesPerRow: width, space: colorSpace, bitmapInfo: 0) else { return nil }
        imageContext.draw(cgImage, in: CGRect(origin: CGPoint.zero, size: CGSize(width: width, height: height)))

        return UnsafeMutableBufferPointer<UInt32>(start: data, count: totalBytes)
    }

    public func normalizedGrayScalePixels() -> [CGFloat]?
    {
        guard let cgImage = self.cgImage else { return nil }
        guard let pixels = self.grayScalePixels() else { return nil }

        let width = cgImage.width
        let height = cgImage.height
        var result = [CGFloat]()

        for y in 0..<height
        {
            for x in 0..<width
            {
                let index = ((width * y) + x)
                let pixel = (CGFloat(pixels[index]) / 255.0)
                result.append(pixel)
            }
        }

        return result
    }
}

绘制数字的代码:

    func drawLineFrom(fromPoint: CGPoint, toPoint: CGPoint)
    {
        UIGraphicsBeginImageContextWithOptions(self.view.bounds.size, false, 1)

        self.tempImageView.image?.draw(at: CGPoint.zero)

        let context = UIGraphicsGetCurrentContext()
        context?.move(to: fromPoint)
        context?.addLine(to: toPoint)
        context?.setLineCap(.round)
        context?.setLineWidth(self.brushWidth)
        context?.setStrokeColor(gray: 0, alpha: 1)
        context?.strokePath()

        self.tempImageView.image = UIGraphicsGetImageFromCurrentImageContext()
        self.tempImageView.alpha = self.opacity

        UIGraphicsEndImageContext()
    }

    override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?)
    {
        self.swiped = false

        if let touch = touches.first {
            self.lastPoint = touch.location(in: self.view)
        }
    }

    override func touchesMoved(_ touches: Set<UITouch>, with event: UIEvent?)
    {
        self.swiped = true

        if let touch = touches.first
        {
            let currentPoint = touch.location(in: self.view)
            self.drawLineFrom(fromPoint: self.lastPoint, toPoint: currentPoint)

            self.lastPoint = currentPoint
        }
    }

    override func touchesEnded(_ touches: Set<UITouch>, with event: UIEvent?)
    {
        if !swiped {
            self.drawLineFrom(fromPoint: self.lastPoint, toPoint: self.lastPoint)
        }

        self.predictionLabel.text = "Predication: \(self.predict())"

        self.tempImageView.image = nil
    }

预测号码的代码:

    private func printNumber(rowSize: Int, inputs: Vector)
    {
        for (index, pixel) in inputs.enumerated()
        {
            if index % rowSize == 0 { print() }

            if (pixel > 0) {
                print("1", terminator: " ")
            }
            else { print(".", terminator: " ") }
        }

        print()
    }

    private func predict() -> Scalar
    {
        let resizedImaege = self.tempImageView.image!.resizedImage(CGSize(width: 28, height: 28), interpolationQuality: .high)
        let inputs = resizedImaege!.normalizedGrayScalePixels()!.flatMap({ Scalar($0) })
        self.feedforwardResult = self.neuralNetwork!.feedForward(inputs: inputs)

        self.printNumber(rowSize: 28, inputs: inputs)

        let max = self.feedforwardResult!.activations.last!.max()!
        let prediction = self.feedforwardResult!.activations.last!.index(of: max)!
        return Scalar(prediction)
    }

1 个答案:

答案 0 :(得分:2)

你的代码中有一件非常糟糕的事情就是这一行:

    defer { data.deallocate(capacity: totalBytes) }

data.deallocate(capacity: totalBytes)在退出方法grayScalePixels()之前执行。因此,返回的baseAddress的{​​{1}}指向已经解除分配的区域,这意味着在访问该区域时您无法预期任何可预测的结果。

如果你想使用UnsafeMutableBufferPointer,你需要在完成对它的所有访问后解除分配区域(下面代码中的#1):

UnsafeMutableBufferPointer
在当前的Swift实现中,private func grayScalePixels() -> UnsafeMutableBufferPointer<UInt8>? { guard let cgImage = self.cgImage else { return nil } let bitsPerComponent = 8 let width = cgImage.width let height = cgImage.height let totalBytes = width * height let colorSpace = CGColorSpaceCreateDeviceGray() let data = UnsafeMutablePointer<UInt8>.allocate(capacity: totalBytes) data.initialize(to: UInt8.max, count: totalBytes) //<- #4 guard let imageContext = CGContext(data: data, width: width, height: height, bitsPerComponent: bitsPerComponent, bytesPerRow: width, space: colorSpace, bitmapInfo: 0) else { return nil } imageContext.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height)) return UnsafeMutableBufferPointer(start: data, count: totalBytes) } public func normalizedGrayScalePixels() -> [CGFloat]? { guard let cgImage = self.cgImage else { return nil } guard let pixels = self.grayScalePixels() else { return nil } let width = cgImage.width let height = cgImage.height var result: [CGFloat] = [] for y in 0..<height { for x in 0..<width { let index = width * y + x let pixel = CGFloat(pixels[index]) / CGFloat(UInt8.max) result.append(pixel) } } pixels.baseAddress!.deinitialize(count: pixels.count) //<- #2 pixels.baseAddress!.deallocate(capacity: pixels.count) //<- #1 return result } 可能不需要{p>(#2)deinitialize,但序列:allocate - initialize - deinitilize - deallocate是推荐的方式。

(触及的其他一些行只是我的偏好,而不是关键。)

或者,如果你想使用Swift UInt8而不是Array,你可以这样写:

UnsafeMutableBufferPointer

您可能需要修改上面的代码,使其与您的代码一起使用,因为我无法使用private func grayScalePixels() -> [UInt8]? { guard let cgImage = self.cgImage else { return nil } let bitsPerComponent = 8 let width = cgImage.width let height = cgImage.height let totalBytes = width * height let colorSpace = CGColorSpaceCreateDeviceGray() var byteArray: [UInt8] = Array(repeating: UInt8.max, count: totalBytes) //<- #4 let success = byteArray.withUnsafeMutableBufferPointer {(buffer)->Bool in guard let imageContext = CGContext(data: buffer.baseAddress!, width: width, height: height, bitsPerComponent: bitsPerComponent, bytesPerRow: width, space: colorSpace, bitmapInfo: 0) else { return false } imageContext.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height)) return true; } return success ? byteArray : nil } public func normalizedGrayScalePixels() -> [CGFloat]? { guard let cgImage = self.cgImage else { return nil } guard let pixels = self.grayScalePixels() else { return nil } let width = cgImage.width let height = cgImage.height var result: [CGFloat] = [] for y in 0..<height { for x in 0..<width { let index = width * y + x let pixel = CGFloat(pixels[index]) / CGFloat(UInt8.max) result.append(pixel) } } return result } UInt32版本重现相同的结果。

修改

我在代码中发现了一个问题。您的绘图代码与以下内容划线:

grayScalePixels()

灰度0,黑色。在我的旧代码中,我将位图初始化为:

    context?.setStrokeColor(gray: 0, alpha: 1)

或:

    data.initialize(to: 0, count: totalBytes)

因此,在黑色上绘制黑色,结果:全黑,8位灰度,全0。 (我第一次写 var byteArray: [UInt8] = Array(repeating: 0, count: totalBytes) 可能不需要,但这是一个错误。带有alpha的图像将与初始位图内容混合在一起。)

我的更新代码(标有initialize)用白色初始化位图(8位灰度,#4 == 255 == 0xFF)。< / p>

通过更新UInt8.max

,您最好检测到非白色像素
printNumber(rowSize:inputs:)

在标准化灰度等级为float时,private func printNumber(rowSize: Int, inputs: Vector) { for (index, pixel) in inputs.enumerated() { if index % rowSize == 0 { print() } if pixel < 1.0 { //<- #4 print("1", terminator: "") } else { print(".", terminator: "") } } print() } 是白色的值,您最好将非白色显示为1.0。 (或者,找到另一个更好的门槛。)