Question

我有应用程序（Qt + Android），它从Android的Camera（AVC）+ AudioRecorder（AAC）创建实时流，然后使用librtmp库（v 2.4）将编码数据发送到RTMP服务器

AVC MediaCodec主要功能：

public void videoEncode(byte[] data) {
    // Video buffers
    videoCodecInputBuffers = videoMediaCodec.getInputBuffers();
    videoCodecOutputBuffers = videoMediaCodec.getOutputBuffers();

    int inputBufferIndex = videoMediaCodec.dequeueInputBuffer(-1);
    if (inputBufferIndex >= 0) {
        videoInputBuffer = videoCodecInputBuffers[inputBufferIndex];
        videoCodecInputData = YV12toYUV420Planar(data, encWidth * encHeight);
        videoInputBuffer.clear();
        videoInputBuffer.put(videoCodecInputData);
        videoMediaCodec.queueInputBuffer(inputBufferIndex, 0, videoCodecInputData.length, 0, 0);
    }

    // Get AVC/H.264 frame
    int outputBufferIndex = videoMediaCodec.dequeueOutputBuffer(videoBufferInfo, 0);
    while(outputBufferIndex >= 0) {
        videoOutputBuffer = videoCodecOutputBuffers[outputBufferIndex];
        videoOutputBuffer.get(videoCodecOutputData, 0, videoBufferInfo.size);

        // H.264 / AVC header
        if(videoCodecOutputData[0] == 0x00 && videoCodecOutputData[1] == 0x00 && videoCodecOutputData[2] == 0x00 && videoCodecOutputData[3] == 0x01) {

            // I-frame
            boolean keyFrame = false;
            if((videoBufferInfo.flags & MediaCodec.BUFFER_FLAG_SYNC_FRAME) == MediaCodec.BUFFER_FLAG_SYNC_FRAME) {
                resetTimestamp();
                keyFrame = true;
            }

            int currentTimestamp = cameraAndroid.calcTimestamp();
            if(prevTimestamp == currentTimestamp) currentTimestamp++;
            sendVideoData(videoCodecOutputData, videoBufferInfo.size, currentTimestamp, cameraAndroid.calcTimestamp()); // Native C func
            prevTimestamp = currentTimestamp;

            // SPS / PPS sent
            spsPpsFrame = true;
        }

        videoMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
        outputBufferIndex = videoMediaCodec.dequeueOutputBuffer(videoBufferInfo, 0);
    }
}

AAC MediaCodec主要功能：

public void audioEncode(byte[] data) {

    // Audio buffers
    audioCodecInputBuffers = audioMediaCodec.getInputBuffers();
    audioCodecOutputBuffers = audioMediaCodec.getOutputBuffers();

    // Add raw chunk into buffer
    int inputBufferIndex = audioMediaCodec.dequeueInputBuffer(-1);
    if (inputBufferIndex >= 0) {
        audioInputBuffer = audioCodecInputBuffers[inputBufferIndex];
        audioInputBuffer.clear();
        audioInputBuffer.put(data);
        audioMediaCodec.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0);
    }

    // Encode AAC
    int outputBufferIndex = audioMediaCodec.dequeueOutputBuffer(audioBufferInfo, 0),
        audioOutputBufferSize = 0;
    while(outputBufferIndex >= 0) {
        audioOutputBuffer = audioCodecOutputBuffers[outputBufferIndex];
        audioOutputBuffer.get(audioCodecOutputData, 0, audioBufferInfo.size);

        if(spsPpsFrame || esdsChunk) {
            int currentTimestamp = cameraAndroid.calcTimestamp();
            if(prevTimestamp == currentTimestamp) currentTimestamp++;
            sendAudioData(audioCodecOutputData, audioBufferInfo.size, currentTimestamp); // Native C func
            prevTimestamp = currentTimestamp;
            esdsChunk = false;
        }

        // Next chunk
        audioMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
        outputBufferIndex = audioMediaCodec.dequeueOutputBuffer(audioBufferInfo, 0);
    }
}

Camera在setPreviewCallbackWithBuffer和AudioRecorder的其他帖子中编码的帧：

audioThread = new Thread(new Runnable() {
    public void run() {
        audioBufferSize = AudioRecord.getMinBufferSize(44100, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
        while(!audioThread.interrupted()) {
            int ret = mic.read(audioCodecInputData, 0, audioBufferSize);
            if(ret >= 0)
                cameraAndroid.audioEncode(audioCodecInputData);
        }
    }
});

sendVideoData和sendAudioData是本机C函数（librtmp func -s + JNI）：

public synchronized native void sendVideoData(byte[] buf, int size, int timestamp, boolean keyFrame);
public synchronized native void sendAudioData(byte[] buf, int size, int timestamp);

我无法理解的主要问题是：当我从Adobe Flash Player播放时，为什么直播流绝对不稳定？前1-2秒的流是完全正确的，但后来我总是每隔2秒（videoMediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 2)）和非常糟糕的声音流看到I帧，在I帧间隔期间我可以听到毫秒，然后它会中断

有人可以向我展示创建稳定直播的正确方法吗？哪里我错了？

另外，我在这里发布了AVC / AAC MediaCodec设置（这里可能有问题吗？）：

// H.264/AVC (advanced video coding) format
MediaFormat videoMediaFormat = MediaFormat.createVideoFormat("video/avc", encWidth, encHeight);
videoMediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar);
videoMediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, encWidth * encHeight * 4);                        // бит в секунду
videoMediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, fps);                                           // FPS
videoMediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, iFrameInterval);                          // interval секунд между I-frames
videoMediaCodec = MediaCodec.createEncoderByType("video/avc");
videoMediaCodec.configure(videoMediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);

// AAC (advanced audio coding) format
MediaFormat audioMediaFormat = MediaFormat.createAudioFormat("audio/mp4a-latm", 44100, 1);              // mime-type, sample rate, channel count
audioMediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 64 * 1000);                                       // kbps
audioMediaFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
audioMediaFormat.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, audioBufferSize);                           // 4096 (default) / 4736 * 1 (min audio buffer size)
audioMediaCodec = MediaCodec.createEncoderByType("audio/mp4a-latm");
audioMediaCodec.configure(audioMediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);

更新我尝试用ffmpeg播放流（感谢@Robert Rowntree）以及我在控制台中不断看到的内容：

输出流0：1中的非单调DTS;上一篇：95054，目前： 46136;更改为95056.这可能会导致时间戳不正确输出文件。

所以，我检查android app的输出，但我看不到错误的行（a - 编码的AAC块，v - 编码的AVC帧，整数值 - 以毫秒为单位的时间戳）：{ {3}}

这是正确的时间戳吗？

Android的MediaCodec（API 16）：AAC + AVC / H.264直播流不稳定

0 个答案: