Question

我正在尝试将YUV_420_888图像转换为位图，来自camera2预览。但输出图像的颜色不正确。

接下来是我正在运行的测试代码来生成位图。仅限测试代码，因此请不要对不相关的因素进行任何代码审查，例如正在回收位图，或者不断创建RenderScript。此代码仅用于测试从YUV到RGB的转换，仅此而已。

其他因素，代码是从API 22及更高版本开始运行的，因此使用特定于RenderScript的 ScriptIntrinsicYuvToRGB 就足够了，而不必使用旧的手动转换，只有在以前的Android版本中才需要缺乏适当的YUV_420_888支持。

由于RenderScript已经提供了专用的ScriptIntrinsicYuvToRGB，用于处理所有类型的YUV转换，我认为问题可能在于如何从Image对象获取YUV字节数据，但我无法确定问题所在

要在Android Studio中查看输出位图，请在bitmap.recycle（）中放置一个断点，因此在它被回收之前，您可以使用“view bitmap”选项在Variables Debug Window中查看它。

如果有人能发现转换有什么问题，请告诉我们：

@Override
public void onImageAvailable(ImageReader reader)
{
    RenderScript rs = RenderScript.create(this.mContext);

    final Image image = reader.acquireLatestImage();

    final Image.Plane[] planes = image.getPlanes();
    final ByteBuffer planeY = planes[0].getBuffer();
    final ByteBuffer planeU = planes[1].getBuffer();
    final ByteBuffer planeV = planes[2].getBuffer();

    // Get the YUV planes data

    final int Yb = planeY.rewind().remaining();
    final int Ub = planeU.rewind().remaining();
    final int Vb = planeV.rewind().remaining();

    final ByteBuffer yuvData = ByteBuffer.allocateDirect(Yb + Ub + Vb);

    planeY.get(yuvData.array(), 0, Yb);
    planeU.get(yuvData.array(), Yb, Vb);
    planeV.get(yuvData.array(), Yb + Vb, Ub);

    // Initialize Renderscript

    Type.Builder yuvType = new Type.Builder(rs, Element.YUV(rs))
            .setX(image.getWidth())
            .setY(image.getHeight())
            .setYuvFormat(ImageFormat.YUV_420_888);

    final Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs))
            .setX(image.getWidth())
            .setY(image.getHeight());

    Allocation yuvAllocation = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT);
    Allocation rgbAllocation = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT);

    // Convert

    yuvAllocation.copyFromUnchecked(yuvData.array());

    ScriptIntrinsicYuvToRGB scriptYuvToRgb = ScriptIntrinsicYuvToRGB.create(rs, Element.YUV(rs));
    scriptYuvToRgb.setInput(yuvAllocation);
    scriptYuvToRgb.forEach(rgbAllocation);

    // Get the bitmap

    Bitmap bitmap = Bitmap.createBitmap(image.getWidth(), image.getHeight(), Bitmap.Config.ARGB_8888);
    rgbAllocation.copyTo(bitmap);

    // Release

    bitmap.recycle();

    yuvAllocation.destroy();
    rgbAllocation.destroy();
    rs.destroy();

    image.close();
}

Answer 1

回答我自己的问题，实际问题就像我怀疑我是如何将Image平面转换为ByteBuffer。接下来的解决方案，应该适用于NV21和YV12。由于YUV数据已经出现在不同的平面上，因此只需根据行和像素的步幅来获得正确的方法。还需要对数据如何传递给RenderScript内在函数进行一些小修改。

注意：对于生产优化onImageAvailable（）不间断流的生产，相反，在进行转换之前，应将Image字节数据复制到单独的缓冲区中，并在单独的线程中执行转换（根据您的要求）。但由于这不是问题的一部分，因此在下一个代码中，转换直接放在onImageAvailable（）中以简化答案。如果有人需要知道如何复制图像数据，请创建一个新问题并告诉我，以便我分享我的代码。

@Override
public void onImageAvailable(ImageReader reader)
{
    // Get the YUV data

    final Image image = reader.acquireLatestImage();
    final ByteBuffer yuvBytes = this.imageToByteBuffer(image);

    // Convert YUV to RGB

    final RenderScript rs = RenderScript.create(this.mContext);

    final Bitmap        bitmap     = Bitmap.createBitmap(image.getWidth(), image.getHeight(), Bitmap.Config.ARGB_8888);
    final Allocation allocationRgb = Allocation.createFromBitmap(rs, bitmap);

    final Allocation allocationYuv = Allocation.createSized(rs, Element.U8(rs), yuvBytes.array().length);
    allocationYuv.copyFrom(yuvBytes.array());

    ScriptIntrinsicYuvToRGB scriptYuvToRgb = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));
    scriptYuvToRgb.setInput(allocationYuv);
    scriptYuvToRgb.forEach(allocationRgb);

    allocationRgb.copyTo(bitmap);

    // Release

    bitmap.recycle();

    allocationYuv.destroy();
    allocationRgb.destroy();
    rs.destroy();

    image.close();
}

private ByteBuffer imageToByteBuffer(final Image image)
{
    final Rect crop   = image.getCropRect();
    final int  width  = crop.width();
    final int  height = crop.height();

    final Image.Plane[] planes     = image.getPlanes();
    final byte[]        rowData    = new byte[planes[0].getRowStride()];
    final int           bufferSize = width * height * ImageFormat.getBitsPerPixel(ImageFormat.YUV_420_888) / 8;
    final ByteBuffer    output     = ByteBuffer.allocateDirect(bufferSize);

    int channelOffset = 0;
    int outputStride = 0;

    for (int planeIndex = 0; planeIndex < 3; planeIndex++)
    {
        if (planeIndex == 0)
        {
            channelOffset = 0;
            outputStride = 1;
        }
        else if (planeIndex == 1)
        {
            channelOffset = width * height + 1;
            outputStride = 2;
        }
        else if (planeIndex == 2)
        {
            channelOffset = width * height;
            outputStride = 2;
        }

        final ByteBuffer buffer      = planes[planeIndex].getBuffer();
        final int        rowStride   = planes[planeIndex].getRowStride();
        final int        pixelStride = planes[planeIndex].getPixelStride();

        final int shift         = (planeIndex == 0) ? 0 : 1;
        final int widthShifted  = width >> shift;
        final int heightShifted = height >> shift;

        buffer.position(rowStride * (crop.top >> shift) + pixelStride * (crop.left >> shift));

        for (int row = 0; row < heightShifted; row++)
        {
            final int length;

            if (pixelStride == 1 && outputStride == 1)
            {
                length = widthShifted;
                buffer.get(output.array(), channelOffset, length);
                channelOffset += length;
            }
            else
            {
                length = (widthShifted - 1) * pixelStride + 1;
                buffer.get(rowData, 0, length);

                for (int col = 0; col < widthShifted; col++)
                {
                    output.array()[channelOffset] = rowData[col * pixelStride];
                    channelOffset += outputStride;
                }
            }

            if (row < heightShifted - 1)
            {
                buffer.position(buffer.position() + rowStride - length);
            }
        }
    }

    return output;
}

Answer 2

RenderScript支持YUV_420_888作为ScriptIntrinsicYuvToRGB的源代码

创建分配和ScriptIntrinsicYuvToRGB

RenderScript renderScript = RenderScript.create(this);
ScriptIntrinsicYuvToRGB mScriptIntrinsicYuvToRGB = ScriptIntrinsicYuvToRGB.create(renderScript, Element.YUV(renderScript));
Allocation mAllocationInYUV = Allocation.createTyped(renderScript, new Type.Builder(renderScript, Element.YUV(renderScript)).setYuvFormat(ImageFormat.YUV_420_888).setX(480).setY(640).create(), Allocation.USAGE_IO_INPUT | Allocation.USAGE_SCRIPT);
Allocation mAllocationOutRGB = Allocation.createTyped(renderScript, Type.createXY(renderScript, Element.RGBA_8888(renderScript), 480, 640), Allocation.USAGE_SCRIPT | Allocation.USAGE_IO_OUTPUT);

设置Allocation.getSurface（）以从相机接收图像数据

final CaptureRequest.Builder captureRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
captureRequest.addTarget(mAllocationInYUV.getSurface());

输出到TextureView或ImageReader或SurfaceView

mAllocationOutRGB.setSurface(new Surface(mTextureView.getSurfaceTexture()));
mAllocationInYUV.setOnBufferAvailableListener(new Allocation.OnBufferAvailableListener() {
    @Override
    public void onBufferAvailable(Allocation a) {
        a.ioReceive();
        mScriptIntrinsicYuvToRGB.setInput(a);
        mScriptIntrinsicYuvToRGB.forEach(mAllocationOutRGB);
        mAllocationOutRGB.ioSend();
    }
});

Answer 3

没有直接的方法将YUV_420_888相机帧复制到RS分配中。实际上，截至今天，Renderscript does not support this format。

如果您知道，在引擎盖下，您的帧是NV21或YV12 - 您可以将整个ByteBuffer复制到数组并将其传递给RS Allocation。

在Android camera2

3 个答案: