Question

编辑：没关系，我解决了我的问题。

// --------------------------------------------- ---------------------------------

我在尝试使用Android MediaCodec API进行MJPEG流解码时遇到了一些问题。我尝试做的是从我的网络摄像头流式传输jpeg帧并在我的Android设备上使用硬件加速解码显示它们（目前我使用的是Nvidia Shield Tablet，Android 6.0）。

现在我正处于收到jpeg框架的位置，我可以使用 BitmapFactory.decodeStream（）功能在屏幕上显示它。但是，我想要做的是在专用硬件上解码图像，这样我就可以获得更好的性能和更好的电池寿命。为了做到这一点，我想将MediaCodec API用于＆＃39; video / mjpeg＆＃39;我非常确定可以在我的设备上使用的解码器（我检查了 MediaCodecList ，它正好在名称＆＃39; OMX.Nvidia.mjpeg.decoder＆＃39; ）。

我将我的MediaCodec与SurfaceView一起使用，就像它在Grafika＆＃39;中所做的一样。回购。（https://github.com/google/grafika）

这是在 surfaceCreated（）回调中调用的解码器初始化代码，完成时没有任何例外：

private Surface testSurface; 
private MediaCodec decoder;

//....

private void initMediaApi() throws IOException {
    decoder = MediaCodec.createDecoderByType("video/mjpeg");

    MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/mjpeg", 720, 480);
    mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 25);

    decoder.configure(mediaFormat, testSurface, null, 0);
    decoder.start();
}

之后我有一个函数，每次收到一个新的jpeg帧时都会调用它。它需要两个参数：一个带有jpeg图像的字节数组和该图像的字节大小：

private void testMediaApi(final byte[] frameBuffer, final int length) throws IOException {

    final int TIMEOUT_USEC = 10000;

    long firstInputTimeNsec = System.nanoTime();

    int inputBufIndex = decoder.dequeueInputBuffer(TIMEOUT_USEC);
    if (inputBufIndex < 0) {
        return;
    }

    ByteBuffer inputBuffer = decoder.getInputBuffer(inputBufIndex);
    inputBuffer.clear();
    inputBuffer.put(frameBuffer, 0, length);

    decoder.queueInputBuffer(inputBufIndex, 0, length, 0, 0);

    boolean outputDone = false;

    while (!outputDone) {
        outputDone = true;

        MediaCodec.BufferInfo mBufferInfo = new MediaCodec.BufferInfo();
        int decoderStatus = decoder.dequeueOutputBuffer(mBufferInfo, 1000000);
        Log.d(LOG_TAG, "decoderStatus: " + decoderStatus);

        if (decoderStatus == MediaCodec.INFO_TRY_AGAIN_LATER) {
            // no output available yet
            Log.d(TAG, "no output from decoder available");
        } else if (decoderStatus == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
            // not important for us, since we're using Surface
            Log.d(TAG, "decoder output buffers changed");
        } else if (decoderStatus == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
            MediaFormat newFormat = decoder.getOutputFormat();
            Log.d(TAG, "decoder output format changed: " + newFormat);
        } else if (decoderStatus < 0) {
            throw new RuntimeException("unexpected result from decoder.dequeueOutputBuffer: " + decoderStatus);
        } else { // decoderStatus >= 0
            if (firstInputTimeNsec != 0) {
                // Log the delay from the first buffer of input to the first buffer
                // of output.
                long nowNsec = System.nanoTime();
                Log.d(TAG, "startup lag " + ((nowNsec - firstInputTimeNsec) / 1000000.0) + " ms");
                firstInputTimeNsec = 0;
            }

            Log.d(TAG, "surface decoder given buffer " + decoderStatus + " (size=" + mBufferInfo.size + ")");
            if ((mBufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                Log.d(TAG, "output EOS");
                outputDone = true;
            }

            boolean doRender = (mBufferInfo.size != 0);

            // As soon as we call releaseOutputBuffer, the buffer will be forwarded
            // to SurfaceTexture to convert to a texture.  We can't control when it
            // appears on-screen, but we can manage the pace at which we release
            // the buffers.
            decoder.releaseOutputBuffer(decoderStatus, doRender);
            Log.d(TAG, "Reached EOS, looping");
            decoder.flush();    // reset decoder state
        }
    }
}

正如您所看到的，我等待1秒钟才完成一帧解码，这应该是充足的。不幸的是，在第一次调用后，我收到解码器状态码-1，这意味着输出数据仍然不可用。在第二帧排队后，我得到状态-3＆＃39;解码器输出缓冲区已更改＆＃39;然后整个事情崩溃给了我这个例外：

07-20 21:21:25.512 10096-10109/piotrek.androidfpvtest W/MediaCodec: ResourceManagerService died.
07-20 21:21:25.513 10096-10127/piotrek.androidfpvtest E/ACodec: OMX/mediaserver died, signalling error!
07-20 21:21:25.513 10096-10127/piotrek.androidfpvtest E/ACodec: signalError(omxError 0x8000100d, internalError -32)
07-20 21:21:25.513 10096-10126/piotrek.androidfpvtest E/MediaCodec: Codec reported err 0xffffffe0, actionCode 0, while in state 6
07-20 21:21:25.513 10096-10125/piotrek.androidfpvtest W/System.err: java.lang.IllegalStateException
07-20 21:21:25.514 10096-10125/piotrek.androidfpvtest W/System.err:     at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
07-20 21:21:25.514 10096-10125/piotrek.androidfpvtest W/System.err:     at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:2622)
07-20 21:21:25.514 10096-10125/piotrek.androidfpvtest W/System.err:     at piotrek.androidfpvtest.MainActivity.testMediaApi(MainActivity.java:226)
07-20 21:21:25.514 10096-10125/piotrek.androidfpvtest W/System.err:     at piotrek.androidfpvtest.MainActivity.doReceive(MainActivity.java:161)
07-20 21:21:25.514 10096-10125/piotrek.androidfpvtest W/System.err:     at piotrek.androidfpvtest.MainActivity.access$000(MainActivity.java:29)
07-20 21:21:25.514 10096-10125/piotrek.androidfpvtest W/System.err:     at piotrek.androidfpvtest.MainActivity$1.run(MainActivity.java:54)
07-20 21:21:25.514 10096-10125/piotrek.androidfpvtest W/System.err:     at java.lang.Thread.run(Thread.java:818)

如果不是连续排队帧，而是等待第一个帧被解码，程序进入无限循环，输出如下：

07-20 21:58:29.970 10577-10641/piotrek.androidfpvtest I/OMXClient: Using client-side OMX mux.
07-20 21:58:30.110 10577-10640/piotrek.androidfpvtest I/MediaCodec: [OMX.Nvidia.mjpeg.decoder] setting surface generation to 10830849
07-20 21:58:30.115 10577-10641/piotrek.androidfpvtest I/ACodec: Enable timestamp filtering for Video Decoder
07-20 21:58:30.126 10577-10641/piotrek.androidfpvtest D/SurfaceUtils: set up nativeWindow 0xae47c708 for 720x480, color 0x106, rotation 0, usage 0x2b00
07-20 21:58:56.719 10577-10641/piotrek.androidfpvtest D/SurfaceUtils: set up nativeWindow 0xae47c708 for 720x480, color 0x147, rotation 0, usage 0x2b00
07-20 21:58:56.720 10577-10641/piotrek.androidfpvtest W/ACodec: [OMX.Nvidia.mjpeg.decoder] setting nBufferCountActual to 11 failed: -1010
07-20 21:58:56.720 10577-10641/piotrek.androidfpvtest W/ACodec: [OMX.Nvidia.mjpeg.decoder] setting nBufferCountActual to 10 failed: -1010
07-20 21:58:57.713 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:57.713 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:58:57.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -3
07-20 21:58:57.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoder output buffers changed
07-20 21:58:58.714 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:58.715 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:58:59.717 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:58:59.718 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:00.720 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:00.720 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:01.721 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:01.721 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:02.723 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:02.723 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:03.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:03.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:04.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:04.724 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:05.726 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:05.726 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available
07-20 21:59:06.727 10577-10639/piotrek.androidfpvtest D/MediaAPITest: decoderStatus: -1
07-20 21:59:06.727 10577-10639/piotrek.androidfpvtest D/MediaAPITest: no output from decoder available

如果有任何人对此API有任何经验，我将非常感谢任何帮助！

Answer 1

我自己解决了我的问题。事实证明，这个特殊的解码器并不支持输出到表面，因此解决方案是获取原始ByteBuffer，然后使用GLES进行YUV-> RGB转换并在屏幕上显示。

Answer 2

MediaCodec.createDecoderByType（“ video / mjpeg”）;会抛出IllegalArgumentException无法初始化video / mjpeg，错误0xfffffffe

Android MediaCodec，解码MJPEG帧

2 个答案: