Question

我正在尝试将frame.capturedDepthData.depthDataMap附加到AVAssetWriterInputPixelBufferAdaptor上，但是结果总是不成功。

我的适配器配置如下：

NSError* error;
videoWriter = [AVAssetWriter.alloc initWithURL:outputURL fileType:AVFileTypeMPEG4 error:&error];
if (error)
{
    NSLog(@"Error creating video writer: %@", error);
    return;
}

NSDictionary* videoSettings = @{
        AVVideoCodecKey: AVVideoCodecTypeH264,
        AVVideoWidthKey: @640,
        AVVideoHeightKey: @360
};

writerInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo outputSettings:videoSettings];
writerInput.transform = CGAffineTransformMakeRotation(M_PI_2);

NSDictionary* sourcePixelBufferAttributesDictionary = @{
        (NSString*) kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_DepthFloat32)
};

adaptor = [AVAssetWriterInputPixelBufferAdaptor
        assetWriterInputPixelBufferAdaptorWithAssetWriterInput:writerInput
                                   sourcePixelBufferAttributes:sourcePixelBufferAttributesDictionary];

if ([videoWriter canAddInput:writerInput])
{
    [videoWriter addInput:writerInput];
}
else
{
    NSLog(@"Error: cannot add writerInput to videoWriter.");
}

[videoWriter startWriting];
[videoWriter startSessionAtSourceTime:kCMTimeZero];

，然后在每个session:(ARSession*)session didUpdateFrame:(ARFrame*)frame回调中，我尝试像这样添加深度像素缓冲区：

if (!adaptor.assetWriterInput.readyForMoreMediaData)
{
    NSLog(@"Asset input writer is not ready for more media data!");
}
else
{
    if (frame.capturedDepthData.depthDataMap != NULL)
    {
        frameCount++;
        CVPixelBufferRef pixelRef = frame.capturedDepthData.depthDataMap;
        BOOL result = [adaptor appendPixelBuffer:frame.capturedDepthData.depthDataMap withPresentationTime:CMTimeMake(frameCount, 15)];
    }
}

但是附加像素缓冲区的结果始终为FALSE。

现在，如果我尝试将frame.capturedImage附加到正确配置的适配器上，那将永远成功，这就是我目前使用前置摄像头制作视频文件的方式。

但是我不知道如何从深度像素缓冲区制作视频？

Answer 1

以下是如何将depthDataMap像素缓冲区转换为可附加到适配器的有效像素缓冲区的示例：

- (void) session:(ARSession*)session didUpdateFrame:(ARFrame*)frame
{
    CVPixelBufferRef depthDataMap = frame.capturedDepthData.depthDataMap;

    if(!depthDataMap)
    {
        // no depth data available
        return;
    }

    CIImage* image = [CIImage imageWithCVPixelBuffer:depthDataMap];
    CVPixelBufferRef buffer = NULL;
    CVReturn err = PixelBufferCreateFromImage(image, &buffer);

    [adaptorDepth appendPixelBuffer:buffer withPresentationTime:CMTimeMake(frameDepthCount, 15)] // 15 is number of fps
}


CVReturn PixelBufferCreateFromImage(CIImage* ciImage, CVPixelBufferRef* outBuffer) {
    CIContext* context = [CIContext context];

    NSDictionary* attributes = @{ (NSString*) kCVPixelBufferCGBitmapContextCompatibilityKey: @YES,
                                  (NSString*) kCVPixelBufferCGImageCompatibilityKey: @YES
    };

    CVReturn err = CVPixelBufferCreate(kCFAllocatorDefault,
                                       (size_t) ciImage.extent.size.width, (size_t) ciImage.extent.size.height,
                                       kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef _Nullable) (attributes),
                                       outBuffer);
    if (err)
    {
        return err;
    }

    if (outBuffer)
    {
        [context render:ciImage toCVPixelBuffer:*outBuffer];
    }

    return kCVReturnSuccess;
}

该键位于PixelBufferCreateFromImage方法中，该方法能够从原始深度像素缓冲区的CIImage创建一个有效的像素缓冲区。

如何从ARKit人脸会话深度像素缓冲区创建视频

1 个答案: