Question

我正在使用MediaCodec将Android摄像头流编码为h264格式，然后使用gstreamer通过gst-rtsp-server创建RTSP流。我可以使用gstreamer客户端管道播放流。但是无法使用vlc播放。任何人都可以帮我在vlc上播放流吗？

这是MediaCodec代码，

final int TIMEOUT_USEC = 10000;     // no timeout -- check for buffers, bail if none
    try {
        ByteBuffer[] encoderOutputBuffers = encoder.getOutputBuffers();
        bufferInfo = new MediaCodec.BufferInfo();

            while (true) {
                int encoderStatus = encoder.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);
                if (encoderStatus == MediaCodec.INFO_TRY_AGAIN_LATER) {
                    // no output available yet
                    break;
                } else if (encoderStatus == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
                    // not expected for an encoder
                    encoderOutputBuffers = encoder.getOutputBuffers();
                } else if (encoderStatus == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
                    // Should happen before receiving buffers, and should only happen once.
                    // The MediaFormat contains the csd-0 and csd-1 keys, which we'll need
                    // for MediaMuxer.  It's unclear what else MediaMuxer might want, so
                    // rather than extract the codec-specific data and reconstruct a new
                    // MediaFormat later, we just grab it here and keep it around.

                    // The PPS and PPS shoud be there

                    if (muxerStarted) {
                        throw new RuntimeException("format changed twice");
                    }

                    muxerStarted = true;
                } else if (encoderStatus < 0) {
                    Log.w(TAG, "unexpected result from encoder.dequeueOutputBuffer: " + encoderStatus);
                    // let's ignore it
                } else {
                    ByteBuffer encodedData = encoderOutputBuffers[encoderStatus];
                    if (encodedData == null) {
                        throw new RuntimeException("encoderOutputBuffer " + encoderStatus +
                                " was null");
                    }

                    if (bufferInfo.size != 0) {

                        encodedData.position(bufferInfo.offset);
                        encodedData.limit(bufferInfo.offset + bufferInfo.size);


                        byte[] outData = new byte[bufferInfo.size];
                        encodedData.get(outData);

                        if (spsPpsInfo == null) {
                            ByteBuffer spsPpsBuffer = ByteBuffer.wrap(outData);
                            if (spsPpsBuffer.getInt() == 0x00000001) {
                                spsPpsInfo = new byte[outData.length];
                                System.arraycopy(outData, 0, spsPpsInfo, 0, outData.length);
                            } else
                                return;
                        } else {
                            outputStream.write(outData);
                        }
                    }

                    encoder.releaseOutputBuffer(encoderStatus, false);

                    if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                        Log.w(TAG, "reached end of stream unexpectedly");
                        break;      // out of while
                    }
                }
            }

            byte[] ret = outputStream.toByteArray();


            if (ret.length > 5 && ret[4] == 0x65) {
                Log.d(TAG, "----> Setting SPS PPS info");
                byte[] concBuffer = new byte[spsPpsInfo.length + ret.length];
                System.arraycopy(spsPpsInfo, 0, concBuffer, 0, spsPpsInfo.length);
                System.arraycopy(ret, 0, concBuffer, spsPpsInfo.length, ret.length);

                outputStream.reset();
                outputStream.write(concBuffer);
                //outputStream.write(ret, spsPpsInfo.length, (spsPpsInfo.length + ret.length-1));
            }
        } catch (Throwable t) {
            t.printStackTrace();
        }

        byte ret[] = outputStream.toByteArray();
        outputStream.reset();

        // Log.d(TAG, "Got buffer with size " + ret.length + " and needData " + needData);

        if (needData == 1 && ret.length != 0) {
            if (streamMode == Native.STREAM_MODE_RTP) {
                // Log.d(TAG, "Sending buffer to RTP pipeline with size " + ret.length);

                Native.nativeRTPAddStream(ret, ret.length, bufferInfo.presentationTimeUs, native_custom);
            } else if (streamMode == Native.STREAM_MODE_RTSP) {
                // Log.d(TAG, "Sending buffer to RTSP pipeline with size " + ret.length);

                Native.nativeRTSPAddStream(ret, ret.length, bufferInfo.presentationTimeUs, native_custom);
            }
        }

这是gstreamer管道，

appsrc name=camsrc ! h264parse ! rtph264pay name=pay0 pt=96

在上述管道的末尾，附加了gst-rtsp-server。

Answer 1

问题在于缓冲时间戳（bufferInfo.presentationTimeUs）。 Android相机发送的时间戳错误。在将每个缓冲区发送到GStreamer接收器之前，您需要对其进行重新定时。

执行此操作的一种方法是使用identity元素访问每个缓冲区。另一种方法是通过手动计算每个缓冲区的时间戳来纠正Android本身的时间戳。

GStreamer中的重新加时间公式看起来像这样

GstClockTime time = gst_util_uint64_scale_int (1, GST_SECOND, absCurFps);
GST_BUFFER_PTS (buffer) = GST_BUFFER_DTS (buffer) += time;
GST_BUFFER_DURATION (buffer) = 1 * GST_SECOND / absCurFps;

实际上，它假设每个缓冲区都是一帧，并将表示时间戳设置为base_time + 1 / (current_fps)

无法在vlc上播放android h264编码视频

1 个答案: