Question

我正在尝试在捕获的图像中的某个点找到深度数据，并在meters中返回距离。

我启用了深度数据，并且正在捕捉图像旁边的数据。我从图像中心的X，Y坐标（当按下时）得到点并使用

将其转换为缓冲区索引

Int((width - touchPoint.x) * (height - touchPoint.y))

WIDTH和HEIGHT是捕获图像的尺寸。我不确定这是否是实现这一目标的正确方法。

我处理深度数据：

func handlePhotoDepthCalculation(point : Int) {

    guard let depth = self.photo else {
        return
    }

    //
    // Convert Disparity to Depth
    //
    let depthData = (depth.depthData as AVDepthData!).converting(toDepthDataType: kCVPixelFormatType_DepthFloat32)
    let depthDataMap = depthData.depthDataMap //AVDepthData -> CVPixelBuffer

    //
    // Set Accuracy feedback
    //
    let accuracy = depthData.depthDataAccuracy
    switch (accuracy) {
    case .absolute:
        /* 
        NOTE - Values within the depth map are absolutely 
        accurate within the physical world.
        */
        self.accuracyLbl.text = "Absolute"
        break
    case .relative:
        /* 
        NOTE - Values within the depth data map are usable for 
        foreground/background separation, but are not absolutely 
        accurate in the physical world. iPhone always produces this.
        */
        self.accuracyLbl.text = "Relative"
    }

    //
    // We convert the data
    //
    CVPixelBufferLockBaseAddress(depthDataMap, CVPixelBufferLockFlags(rawValue: 0))
    let depthPointer = unsafeBitCast(CVPixelBufferGetBaseAddress(depthDataMap), to: UnsafeMutablePointer<Float32>.self)

    //
    // Get depth value for image center
    //
    let distanceAtXYPoint = depthPointer[point]

    //
    // Set UI
    //
    self.distanceLbl.text = "\(distanceAtXYPoint) m" //Returns distance in meters?
    self.filteredLbl.text = "\(depthData.isDepthDataFiltered)" 
}

我不相信我得到了正确的立场。从我的研究来看，看起来精确度仅在.relative或.absolute中返回而不是浮点数/整数？

Answer 1

要在CGPoint上访问深度数据，请执行以下操作：

let point = CGPoint(35,26)
let width = CVPixelBufferGetWidth(depthDataMap)
let distanceAtXYPoint = depthPointer[Int(point.y * CGFloat(width) + point.x)]

我希望它有效。

Answer 2

表示深度数据图的一般精度的值。

深度数据图的准确性高度依赖于用于生成它的相机校准数据。如果在捕获时不能精确地确定相机的焦距，则将引入z（深度）平面中的缩放误差。如果在捕获时不能精确确定摄像机的光学中心，则将引入主点误差，导致视差估计中的偏移误差。 这些值会报告地图值与其报告单位的准确性。

案例相对

深度数据图中的值可用于前景/背景分离，但在物理世界中并非绝对准确。

案例绝对

深度图中的值在物理世界中绝对准确。

您可以从AVDepthData缓冲区获取CGPoint，如高度和宽度，如跟随代码。

// Useful data
 let width = CVPixelBufferGetWidth(depthDataMap) 
 let height = CVPixelBufferGetHeight(depthDataMap)

Answer 3

在 Apple 的示例 project 中，他们使用以下代码。

Texturepoint 是投影到示例项目中使用的金属视图的接触点。

// scale
let scale = CGFloat(CVPixelBufferGetWidth(depthFrame)) / CGFloat(CVPixelBufferGetWidth(videoFrame))
let depthPoint = CGPoint(x: CGFloat(CVPixelBufferGetWidth(depthFrame)) - 1.0 - texturePoint.x * scale, y: texturePoint.y * scale)
        
assert(kCVPixelFormatType_DepthFloat16 == CVPixelBufferGetPixelFormatType(depthFrame))
CVPixelBufferLockBaseAddress(depthFrame, .readOnly)
let rowData = CVPixelBufferGetBaseAddress(depthFrame)! + Int(depthPoint.y) * CVPixelBufferGetBytesPerRow(depthFrame)
// swift does not have an Float16 data type. Use UInt16 instead, and then translate
var f16Pixel = rowData.assumingMemoryBound(to: UInt16.self)[Int(depthPoint.x)]
CVPixelBufferUnlockBaseAddress(depthFrame, .readOnly)
        
var f32Pixel = Float(0.0)
var src = vImage_Buffer(data: &f16Pixel, height: 1, width: 1, rowBytes: 2)
var dst = vImage_Buffer(data: &f32Pixel, height: 1, width: 1, rowBytes: 4)
vImageConvert_Planar16FtoPlanarF(&src, &dst, 0)
        
// Convert the depth frame format to cm
let depthString = String(format: "%.2f cm", f32Pixel * 100)

Answer 4

在像素位置访问深度数据：

let depthDataMap: CVPixelBuffer = ...
let pixelX: Int = ...
let pixelY: Int = ...

CVPixelBufferLockBaseAddress(self, .readOnly)
let bytesPerRow = CVPixelBufferGetBytesPerRow(depthDataMap)
let baseAddress = CVPixelBufferGetBaseAddress(depthDataMap)!
assert(kCVPixelFormatType_DepthFloat32 == CVPixelBufferGetPixelFormatType(depthDataMap))

let rowData = baseAddress + pixelY * bytesPerRow
let distance = rowData.assumingMemoryBound(to: Float32.self)[pixelX]

CVPixelBufferUnlockBaseAddress(self, .readOnly)

对我来说，访问深度时不正确和不一致的值

let depthPointer = unsafeBitCast(CVPixelBufferGetBaseAddress(depthDataMap), to: UnsafeMutablePointer<Float32>.self)

如何从AVDepthData缓冲区读取CGPoint的深度数据

4 个答案: