Question

ImageReader从相机预览中获取的每一帧都是格式为YUV_420_888的图像，我想将其用作MLKIT的输入。

在google文档中，我可以输入以下内容运行检测器：

位图
media.Image
ByteBuffer
ByteArray
文件

我尝试将YUV_420_888转换为上述对象，但失败了

Answer 1

我使用此功能将图像帧转换为NV21格式，并在FirebaseVisionImage的元数据中设置图像类型

fun YUV_420_888toNV21(image: ImageProxy): ByteArray {

            val width = image.width
            val height = image.height
            val ySize = width * height
            val uvSize = width * height / 4

            val nv21 = ByteArray(ySize + uvSize * 2)
            val yBuffer = image.planes[0].buffer // Y
            val uBuffer = image.planes[1].buffer // U
            val vBuffer = image.planes[2].buffer // V

            var rowStride = image.planes[0].rowStride
            assert(image.planes[0].pixelStride == 1)

            var pos = 0

            //may need to flip the buffers if you get underflow exception

            if (rowStride == width) { // likely
                yBuffer.get(nv21, 0, ySize)
                pos += ySize

            } else {
                var yBufferPos = (width - rowStride).toLong() // not an actual position
                while (pos < ySize) {
                    yBufferPos = yBufferPos + (rowStride - width).toLong()
                    yBuffer.position(yBufferPos.toInt())
                    yBuffer.get(nv21, pos, width)
                    pos += width
                }
            }

            rowStride = image.planes[2].rowStride
            val pixelStride = image.planes[2].pixelStride

            assert(rowStride == image.planes[1].rowStride)
            assert(pixelStride == image.planes[1].pixelStride)

            if (pixelStride == 2 && rowStride == width && uBuffer.get(0) == vBuffer.get(1)) {
                // maybe V an U planes overlap as per NV21, which means vBuffer[1] is alias of uBuffer[0]
                val savePixel = vBuffer.get(1)
                vBuffer.put(1, 0.toByte())
                if (uBuffer.get(0).toInt() == 0) {
                    vBuffer.put(1, 255.toByte())
                    if (uBuffer.get(0).toInt() == 255) {
                        vBuffer.put(1, savePixel)
                        vBuffer.get(nv21, ySize, uvSize)
                        //Log.d("NV211",DataConverter.jsonify(nv21))
                        return nv21 // shortcut
                    }
                }

                // unfortunately, the check failed. We must save U and V pixel by pixel
                vBuffer.put(1, savePixel)
            }

            // other optimizations could check if (pixelStride == 1) or (pixelStride == 2),
            // but performance gain would be less significant

            for (row in 0 until height / 2) {
                for (col in 0 until width / 2) {
                    val vuPos = col * pixelStride + row * rowStride
                    nv21[pos++] = vBuffer.get(vuPos)
                    nv21[pos++] = uBuffer.get(vuPos)
                }
            }
            //Log.d("NV212",DataConverter.jsonify(nv21))
            return nv21
        }

如何将图像格式YUV_420_888用于Google的MLKIT

1 个答案: