我正在使用Apple VideoTool Box（iOS）压缩设备相机捕获的原始帧。
我的回调是使用包含CMBlockBuffer的CMSampleBufferRef对象调用的。
CMBlockBuffer对象包含H264基本流，但我没有找到任何方法来获取指向基本流的指针。
当我在控制台中打印出CMSampleBufferRef对象时：
(lldb) po blockBufferRef
CMBlockBuffer 0x1701193e0 totalDataLength: 4264 retainCount: 1 allocator: 0x1957c2c80 subBlockCapacity: 2
[0] 4264 bytes @ offset 128 Buffer Reference:
CMBlockBuffer 0x170119350 totalDataLength: 4632 retainCount: 1 allocator: 0x1957c2c80 subBlockCapacity: 2
[0] 4632 bytes @ offset 0 Memory Block 0x10295c000, 4632 bytes (custom V=0 A=0x0 F=0x18498bb44 R=0x0)
似乎我设法获取指针的CMBlockBuffer对象是另一个无法访问的CMBlockBuferRef（4632字节）。
有人可以发布如何访问H264 elemantry流吗？
谢谢！
答案 0 :(得分：41)
我自己一直在努力解决这个问题已经有一段时间了，终于把事情搞清楚了。
函数CMBlockBufferGetDataPointer使您可以访问所需的所有数据，但是有一些不太明显的事情需要将它转换为基本流。
CMBlockBuffer中的数据以AVCC格式存储，而基本流通常遵循附件B规范（here是两种格式的绝佳概述）。在AVCC格式中，4个第一个字节包含NAL单元的长度（H264包的另一个字）。您需要将此标头替换为4字节起始码：0x00 0x00 0x00 0x01，它用作附件B基本流中NAL单元之间的分隔符（3字节版本0x00 0x00 0x01也能正常工作）。
下一个不太明显的事情是单个CMBlockBuffer有时会包含多个NAL单元。 Apple似乎在每个I-Frame NAL单元（也称为IDR）中添加了一个包含元数据的附加NAL单元（SEI）。这可能是您在单个CMBlockBuffer对象中看到多个缓冲区的原因。但是，CMBlockBufferGetDataPointer函数为您提供了一个可以访问所有数据的指针。话虽如此，多个NAL单元的存在使AVCC头的转换复杂化。现在你实际上必须读取AVCC头中包含的长度值以找到下一个NAL单元，并继续转换头，直到你到达缓冲区的末尾。
下一个不太明显的事情是AVCC标题以Big-Endian格式存储，而iOS本身就是Little-Endian。因此，当您正在读取AVCC标头中包含的长度值时，首先将其传递给CFSwapInt32BigToHost函数。
最后不太明显的事情是CMBlockBuffer中的数据不包含参数NAL单元SPS和PPS，其包含解码器的配置参数，例如配置文件，级别，分辨率，帧速率。它们作为元数据存储在样本缓冲区的格式描述中，可以通过函数CMVideoFormatDescriptionGetH264ParameterSetAtIndex访问。请注意，您必须在发送之前将起始代码添加到这些NAL单元。 SPS和PPS NAL单元不必与每个新帧一起发送。解码器只需要读取一次，但通常会定期重新发送它们，例如在每个新的I帧NAL单元之前。
以下是考虑所有这些因素的代码示例。
static void videoFrameFinishedEncoding(void *outputCallbackRefCon,
void *sourceFrameRefCon,
OSStatus status,
VTEncodeInfoFlags infoFlags,
CMSampleBufferRef sampleBuffer) {
// Check if there were any errors encoding
if (status != noErr) {
NSLog(@"Error encoding video, err=%lld", (int64_t)status);
return;
}
// In this example we will use a NSMutableData object to store the
// elementary stream.
NSMutableData *elementaryStream = [NSMutableData data];
// Find out if the sample buffer contains an I-Frame.
// If so we will write the SPS and PPS NAL units to the elementary stream.
BOOL isIFrame = NO;
CFArrayRef attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, 0);
if (CFArrayGetCount(attachmentsArray)) {
CFBooleanRef notSync;
CFDictionaryRef dict = CFArrayGetValueAtIndex(attachmentsArray, 0);
BOOL keyExists = CFDictionaryGetValueIfPresent(dict,
kCMSampleAttachmentKey_NotSync,
(const void **)¬Sync);
// An I-Frame is a sync frame
isIFrame = !keyExists || !CFBooleanGetValue(notSync);
}
// This is the start code that we will write to
// the elementary stream before every NAL unit
static const size_t startCodeLength = 4;
static const uint8_t startCode[] = {0x00, 0x00, 0x00, 0x01};
// Write the SPS and PPS NAL units to the elementary stream before every I-Frame
if (isIFrame) {
CMFormatDescriptionRef description = CMSampleBufferGetFormatDescription(sampleBuffer);
// Find out how many parameter sets there are
size_t numberOfParameterSets;
CMVideoFormatDescriptionGetH264ParameterSetAtIndex(description,
0, NULL, NULL,
&numberOfParameterSets,
NULL);
// Write each parameter set to the elementary stream
for (int i = 0; i < numberOfParameterSets; i++) {
const uint8_t *parameterSetPointer;
size_t parameterSetLength;
CMVideoFormatDescriptionGetH264ParameterSetAtIndex(description,
i,
¶meterSetPointer,
¶meterSetLength,
NULL, NULL);
// Write the parameter set to the elementary stream
[elementaryStream appendBytes:startCode length:startCodeLength];
[elementaryStream appendBytes:parameterSetPointer length:parameterSetLength];
}
}
// Get a pointer to the raw AVCC NAL unit data in the sample buffer
size_t blockBufferLength;
uint8_t *bufferDataPointer = NULL;
CMBlockBufferGetDataPointer(CMSampleBufferGetDataBuffer(sampleBuffer),
0,
NULL,
&blockBufferLength,
(char **)&bufferDataPointer);
// Loop through all the NAL units in the block buffer
// and write them to the elementary stream with
// start codes instead of AVCC length headers
size_t bufferOffset = 0;
static const int AVCCHeaderLength = 4;
while (bufferOffset < blockBufferLength - AVCCHeaderLength) {
// Read the NAL unit length
uint32_t NALUnitLength = 0;
memcpy(&NALUnitLength, bufferDataPointer + bufferOffset, AVCCHeaderLength);
// Convert the length value from Big-endian to Little-endian
NALUnitLength = CFSwapInt32BigToHost(NALUnitLength);
// Write start code to the elementary stream
[elementaryStream appendBytes:startCode length:startCodeLength];
// Write the NAL unit without the AVCC length header to the elementary stream
[elementaryStream appendBytes:bufferDataPointer + bufferOffset + AVCCHeaderLength
length:NALUnitLength];
// Move to the next NAL unit in the block buffer
bufferOffset += AVCCHeaderLength + NALUnitLength;
}
}
答案 1 :(得分：4)
感谢Anton提供了出色的答案！在为基于Swift的项目中直接使用这里讨论的概念的人们提供一个天真的Swift端口。
public func didEncodeFrame(frame: CMSampleBuffer)
{
print ("Received encoded frame in delegate...")
//----AVCC to Elem stream-----//
var elementaryStream = NSMutableData()
//1. check if CMBuffer had I-frame
var isIFrame:Bool = false
let attachmentsArray:CFArray = CMSampleBufferGetSampleAttachmentsArray(frame, false)!
//check how many attachments
if ( CFArrayGetCount(attachmentsArray) > 0 ) {
let dict = CFArrayGetValueAtIndex(attachmentsArray, 0)
let dictRef:CFDictionaryRef = unsafeBitCast(dict, CFDictionaryRef.self)
//get value
let value = CFDictionaryGetValue(dictRef, unsafeBitCast(kCMSampleAttachmentKey_NotSync, UnsafePointer<Void>.self))
if ( value != nil ){
print ("IFrame found...")
isIFrame = true
}
}
//2. define the start code
let nStartCodeLength:size_t = 4
let nStartCode:[UInt8] = [0x00, 0x00, 0x00, 0x01]
//3. write the SPS and PPS before I-frame
if ( isIFrame == true ){
let description:CMFormatDescriptionRef = CMSampleBufferGetFormatDescription(frame)!
//how many params
var numParams:size_t = 0
CMVideoFormatDescriptionGetH264ParameterSetAtIndex(description, 0, nil, nil, &numParams, nil)
//write each param-set to elementary stream
print("Write param to elementaryStream ", numParams)
for i in 0..<numParams {
var parameterSetPointer:UnsafePointer<UInt8> = nil
var parameterSetLength:size_t = 0
CMVideoFormatDescriptionGetH264ParameterSetAtIndex(description, i, ¶meterSetPointer, ¶meterSetLength, nil, nil)
elementaryStream.appendBytes(nStartCode, length: nStartCodeLength)
elementaryStream.appendBytes(parameterSetPointer, length: unsafeBitCast(parameterSetLength, Int.self))
}
}
//4. Get a pointer to the raw AVCC NAL unit data in the sample buffer
var blockBufferLength:size_t = 0
var bufferDataPointer: UnsafeMutablePointer<Int8> = nil
CMBlockBufferGetDataPointer(CMSampleBufferGetDataBuffer(frame)!, 0, nil, &blockBufferLength, &bufferDataPointer)
print ("Block length = ", blockBufferLength)
//5. Loop through all the NAL units in the block buffer
var bufferOffset:size_t = 0
let AVCCHeaderLength:Int = 4
while (bufferOffset < (blockBufferLength - AVCCHeaderLength) ) {
// Read the NAL unit length
var NALUnitLength:UInt32 = 0
memcpy(&NALUnitLength, bufferDataPointer + bufferOffset, AVCCHeaderLength)
//Big-Endian to Little-Endian
NALUnitLength = CFSwapInt32(NALUnitLength)
if ( NALUnitLength > 0 ){
print ( "NALUnitLen = ", NALUnitLength)
// Write start code to the elementary stream
elementaryStream.appendBytes(nStartCode, length: nStartCodeLength)
// Write the NAL unit without the AVCC length header to the elementary stream
elementaryStream.appendBytes(bufferDataPointer + bufferOffset + AVCCHeaderLength, length: Int(NALUnitLength))
// Move to the next NAL unit in the block buffer
bufferOffset += AVCCHeaderLength + size_t(NALUnitLength);
print("Moving to next NALU...")
}
}
print("Read completed...")
}