Question

这里的目标是通过视频通过AVCaptureDataOutput创建一个mp4文件，音频录制一个CoreAudio。然后将两者的CMSampleBuffers发送到AVAssetWriter，该AVAssetWriter附带AVAssetWriterInput（AVMediaTypeVideo）和AVAssetWriterInput（AVMediaTypeAudio）

我的音频编码器将AudioBuffer复制到新的CMSampleBuffer，并将其传递给AVAssetWriterInput（AVMediaTypeAudio）。此示例是如何完成AudioBuffer到CMSampleBuffer的转换。 Converstion to CMSampleBuffer

长话短说，它不起作用。视频显示但没有音频。

但是，如果我注释掉视频编码，那么音频会写入文件并发出声音。

从经验告诉我这是一个时间问题。 Converstion to CMSampleBuffer确实显示

   CMSampleTimingInfo timing = { CMTimeMake(1, 44100.0), kCMTimeZero, kCMTimeInvalid };

它生成{0/1 = 0.000}的时间CMTimeCopyDescription，这对我来说似乎完全没错。我试着跟踪渲染的帧并传递帧计数的时间值和时间刻度的采样率，如下所示

   CMSampleTimingInfo timing = { CMTimeMake(1, 44100.0), CMTimeMake(self.frameCount, 44100.0), kCMTimeInvalid };

但没有骰子。看起来更漂亮的CMSampleTimingInfo {107520/44100 = 2.438}，但文件中仍没有音频。

视频CMSampleBuffer会生成类似{65792640630624/1000000000 = 65792.641, rounded}的内容。这告诉我AVCaptureVideoOutput的时间尺度为10亿，可能是纳秒。而我客人的时间价值就像设备时间。我无法找到有关AVCaptureVideoOutput使用的信息。

任何人都有任何有用的指导？我甚至走在正确的轨道上吗？

继承人转换

    CMSampleBufferRef buff = malloc(sizeof(CMSampleBufferRef));
    CMFormatDescriptionRef format = NULL;

    self.frameCount += inNumberFrames;

    CMTime presentationTime = CMTimeMake(self.frameCount, self.pcmASBD.mSampleRate);

    AudioStreamBasicDescription audioFormat = self.pcmASBD;
    CheckError(CMAudioFormatDescriptionCreate(kCFAllocatorDefault,
                                              &audioFormat,
                                              0,
                                              NULL,
                                              0,
                                              NULL,
                                              NULL,
                                              &format),
               "Could not create format from AudioStreamBasicDescription");

    CMSampleTimingInfo timing = { CMTimeMake(1, self.pcmASBD.mSampleRate), presentationTime, kCMTimeInvalid };

    CheckError(CMSampleBufferCreate(kCFAllocatorDefault,
                                    NULL,
                                    false,
                                    NULL,
                                    NULL,
                                    format,
                                    (CMItemCount)inNumberFrames,
                                    1,
                                    &timing,
                                    0,
                                    NULL,
                                    &buff),
               "Could not create CMSampleBufferRef");

    CheckError(CMSampleBufferSetDataBufferFromAudioBufferList(buff,
                                                              kCFAllocatorDefault,
                                                              kCFAllocatorDefault,
                                                              0,
                                                              audioBufferList),
               "Could not set data in CMSampleBufferRef");

    [self.delegate didRenderAudioSampleBuffer:buff];

    CFRelease(buff);

我创建的assetWriters

    func createVideoInputWriter()->AVAssetWriterInput? {
        let numPixels                               = Int(self.size.width * self.size.height)
        let bitsPerPixel:Int                        = 11
        let bitRate                                 = Int64(numPixels * bitsPerPixel)
        let fps:Int                                 = 30
        let settings:[NSObject : AnyObject]         = [
            AVVideoCodecKey                         : AVVideoCodecH264,
            AVVideoWidthKey                         : self.size.width,
            AVVideoHeightKey                        : self.size.height,
            AVVideoCompressionPropertiesKey         : [
                AVVideoAverageBitRateKey            : NSNumber(longLong: bitRate),
                AVVideoMaxKeyFrameIntervalKey       : NSNumber(integer: fps)
            ]
        ]

        var assetWriter:AVAssetWriterInput!
        if self.mainAssetWriter.canApplyOutputSettings(settings, forMediaType:AVMediaTypeVideo) {
            assetWriter                             = AVAssetWriterInput(mediaType:AVMediaTypeVideo, outputSettings:settings)
            assetWriter.expectsMediaDataInRealTime  = true
            if self.mainAssetWriter.canAddInput(assetWriter) {
                self.mainAssetWriter.addInput(assetWriter)
            }
        }
        return assetWriter;
    }

    func createAudioInputWriter()->AVAssetWriterInput? {
        let settings:[NSObject : AnyObject]         = [
            AVFormatIDKey                           : kAudioFormatMPEG4AAC,
            AVNumberOfChannelsKey                   : 2,
            AVSampleRateKey                         : 44100,
            AVEncoderBitRateKey                     : 64000
        ]

        var assetWriter:AVAssetWriterInput!
        if self.mainAssetWriter.canApplyOutputSettings(settings, forMediaType:AVMediaTypeAudio) {
            assetWriter                             = AVAssetWriterInput(mediaType:AVMediaTypeAudio, outputSettings:settings)
            assetWriter.expectsMediaDataInRealTime  = true
            if self.mainAssetWriter.canAddInput(assetWriter) {
                self.mainAssetWriter.addInput(assetWriter)
            } else {
                let error = NSError(domain:CMHDFileEncoder.Domain, code:CMHDFileEncoderErrorCode.CantAddInput.rawValue, userInfo:nil)
                self.errorDelegate.hdFileEncoderError(error)
            }
        } else {
            let error = NSError(domain:CMHDFileEncoder.Domain, code:CMHDFileEncoderErrorCode.CantApplyOutputSettings.rawValue, userInfo:nil)
            self.errorDelegate.hdFileEncoderError(error)
        }
        return assetWriter
    }

Answer 1

当然，问题持续了2个星期，周五晚上发布了问题，并在星期一早上找到解决方案。

我遇到的研究让我走上正轨......

1000000000时标是纳秒。但是时间值必须是设备绝对时间的纳秒。

这篇文章比我更好地解释了 - mach time

我最终使用此代码来修复它

    CMSampleBufferRef buff = malloc(sizeof(CMSampleBufferRef));
    CMFormatDescriptionRef format = NULL;

    AudioStreamBasicDescription audioFormat = self.pcmASBD;
    CheckError(CMAudioFormatDescriptionCreate(kCFAllocatorDefault,
                                              &audioFormat,
                                              0,
                                              NULL,
                                              0,
                                              NULL,
                                              NULL,
                                              &format),
               "Could not create format from AudioStreamBasicDescription");

    uint64_t time = inTimeStamp->mHostTime;
    /* Convert to nanoseconds */
    time *= info.numer;
    time /= info.denom;
    CMTime presentationTime                 = CMTimeMake(time, kDeviceTimeScale);
    CMSampleTimingInfo timing               = { CMTimeMake(1, self.pcmASBD.mSampleRate), presentationTime, kCMTimeInvalid };

    CheckError(CMSampleBufferCreate(kCFAllocatorDefault,
                                    NULL,
                                    false,
                                    NULL,
                                    NULL,
                                    format,
                                    (CMItemCount)inNumberFrames,
                                    1,
                                    &timing,
                                    0,
                                    NULL,
                                    &buff),
               "Could not create CMSampleBufferRef");

    CheckError(CMSampleBufferSetDataBufferFromAudioBufferList(buff,
                                                              kCFAllocatorDefault,
                                                              kCFAllocatorDefault,
                                                              0,
                                                              audioBufferList),
               "Could not set data in CMSampleBufferRef");

使用准确的CMTime将AudioBuffer转换为CMSampleBuffer

1 个答案: