Question

我正在尝试编写某种基于ffMpeg项目的RTSP iOS流媒体。基本上，我从https://github.com/durfu/DFURTSPPlayer得到了这个项目。非常感谢这个项目的创建者，但遗憾的是它在播放流方面存在问题，因此我决定使用一些作者的开发自己编写流媒体播放器。

当然，为了正确播放流媒体，我需要将音频与视频同步，我可以通过比较帧数来实现。但问题在于音频和视频点之间存在巨大差距。我有想法，我得到音频和视频的错误，但我找不到我自己的实际问题。这是功能：

- (void) decodeFrame
{
    int frameFinished = 0;
    AVPacket newPacket;

    while (frameFinished == 0 && av_read_frame(formatCtx, &newPacket) >= 0)
    {
        if (newPacket.stream_index == videoStream)
        {
            avcodec_decode_video2(videoCodecCtx, videoFrame, &frameFinished, &newPacket);
            [self convertFrameToRGB];

            UIImage *newImage = [self currentImage];
            if (newPacket.pts != AV_NOPTS_VALUE) {
                newPacket.pts += av_rescale_q(0, AV_TIME_BASE_Q, formatCtx->streams[audioStream]->time_base);
            }
            unsigned long tempVideoPts = newPacket.pts;                
            double presentTime = tempVideoPts * av_q2d(formatCtx->streams[videoStream]->time_base);
            [_ZFbuffer addNewImage:newImage withPts:tempVideoPts withPresentTime:presentTime];
            NSLog(@"tempVideoPts = %lu", tempVideoPts);
        }
        else if (newPacket.stream_index == audioStream)
        {
            NSMutableData *soundData = [[NSMutableData alloc] initWithBytes:newPacket.data length:newPacket.size];
            if (newPacket.pts != AV_NOPTS_VALUE) {
                newPacket.pts += av_rescale_q(0, AV_TIME_BASE_Q, formatCtx->streams[audioStream]->time_base);
            }
            unsigned long tempAudioPts = newPacket.pts;

            double presentTime = tempAudioPts * av_q2d(formatCtx->streams[audioStream]->time_base);
            [_ZFbuffer addNewSound:soundData withPts:tempAudioPts withPresentTime:presentTime];
            NSLog(@"tempAudioPts = %lu", tempAudioPts);
        }

        av_free_packet(&newPacket);

        if(!streamIsPlaying && [_ZFbuffer bufferedTime:formatCtx->streams[audioStream]->time_base] > 2.0f)  {
            streamIsPlaying = YES;


            //launch audio translation
            if (emptyAudioBuffer != nil) {
                [self unpackNextSound:emptyAudioBuffer];
            }
        }
    }
}

我期待pts或多或少相等，但这是我的日志

2016-01-05 22:03:03.918 DFURTSPPlayer[1569:76194] tempVideoPts = 180000
2016-01-05 22:03:03.959 DFURTSPPlayer[1569:76194] tempVideoPts = 183780
2016-01-05 22:03:04.001 DFURTSPPlayer[1569:76194] tempVideoPts = 187470
2016-01-05 22:03:04.042 DFURTSPPlayer[1569:76194] tempAudioPts = 58368
2016-01-05 22:03:04.042 DFURTSPPlayer[1569:76194] tempVideoPts = 191250
2016-01-05 22:03:04.084 DFURTSPPlayer[1569:76194] tempVideoPts = 195030
2016-01-05 22:03:04.126 DFURTSPPlayer[1569:76194] tempAudioPts = 59392
2016-01-05 22:03:04.126 DFURTSPPlayer[1569:76194] tempVideoPts = 198720
2016-01-05 22:03:04.168 DFURTSPPlayer[1569:76194] tempVideoPts = 202500
2016-01-05 22:03:04.209 DFURTSPPlayer[1569:76194] tempVideoPts = 206280
2016-01-05 22:03:04.250 DFURTSPPlayer[1569:76194] tempAudioPts = 60416
2016-01-05 22:03:04.250 DFURTSPPlayer[1569:76194] tempAudioPts = 61440
2016-01-05 22:03:04.251 DFURTSPPlayer[1569:76194] tempVideoPts = 209970
2016-01-05 22:03:04.292 DFURTSPPlayer[1569:76194] tempVideoPts = 213750
2016-01-05 22:03:04.334 DFURTSPPlayer[1569:76194] tempAudioPts = 62464
2016-01-05 22:03:04.335 DFURTSPPlayer[1569:76194] tempVideoPts = 217530
2016-01-05 22:03:04.375 DFURTSPPlayer[1569:76194] tempVideoPts = 221220
2016-01-05 22:03:04.417 DFURTSPPlayer[1569:76194] tempAudioPts = 63488
2016-01-05 22:03:04.418 DFURTSPPlayer[1569:76194] tempVideoPts = 225000
2016-01-05 22:03:04.459 DFURTSPPlayer[1569:76194] tempVideoPts = 228780
2016-01-05 22:03:04.501 DFURTSPPlayer[1569:76194] tempVideoPts = 232470
2016-01-05 22:03:04.541 DFURTSPPlayer[1569:76194] tempAudioPts = 64512
2016-01-05 22:03:04.542 DFURTSPPlayer[1569:76194] tempVideoPts = 236250
2016-01-05 22:03:04.584 DFURTSPPlayer[1569:76194] tempVideoPts = 240030
2016-01-05 22:03:04.626 DFURTSPPlayer[1569:76194] tempAudioPts = 65536
2016-01-05 22:03:04.626 DFURTSPPlayer[1569:76194] tempVideoPts = 243720
2016-01-05 22:03:04.668 DFURTSPPlayer[1569:76194] tempVideoPts = 247500
2016-01-05 22:03:04.709 DFURTSPPlayer[1569:76194] tempVideoPts = 251280
2016-01-05 22:03:04.750 DFURTSPPlayer[1569:76194] tempAudioPts = 66560
2016-01-05 22:03:04.750 DFURTSPPlayer[1569:76194] tempVideoPts = 254970
2016-01-05 22:03:04.792 DFURTSPPlayer[1569:76194] tempVideoPts = 258750
2016-01-05 22:03:04.834 DFURTSPPlayer[1569:76194] tempAudioPts = 67584
2016-01-05 22:03:04.834 DFURTSPPlayer[1569:76194] tempAudioPts = 68608

如果有人遇到这个问题，请帮帮我。我明白我错过了什么，但我无法理解究竟是什么。感谢您的时间和帮助。

音频和视频点不是同步ffmpeg

0 个答案: