Question

我正在尝试在屏幕上叠加ARFaceAnchor顶点来完成两个场景 1）具有虚拟面部维持位于中心（屏幕上）的位置，但反映几何形状的变化 2）让虚拟面部与实际面部重叠（来自预览层）。

我在这里跟着Rickster's advice，但只是成功地从屏幕上的某个角度投射了脸部（仅显示在左下角并旋转）。我不太熟悉每个矩阵的不同目的，但这是我到目前为止所处的位置。有什么建议吗？

        let modelMatrix = faceAnchor.transform
        var points: [CGPoint] = []

        faceAnchor.geometry.vertices.forEach {
            // Convert the vertex position from model space to camera space (use the anchor’s transform)
            let vertex4 = vector_float4($0.x, $0.y, $0.z, 1)
            let vertexCamera = simd_mul(modelMatrix, vertex4)

            // Multiply with the camera projection with that vector to get to normalized image coordinates
            let normalizedImageCoordinates = simd_mul(projectionMatrix, vertexCamera)

            let point = CGPoint(x: CGFloat(normalizedImageCoordinates.x), y: CGFloat(normalizedImageCoordinates.y))
            points.append(point)
        }

Answer 1

对于那些感兴趣的人，这里是（2）的解决方案 - 你可以将点标准化以保持面部为中心（1）

    let faceAnchors = anchors.flatMap { $0 as? ARFaceAnchor }

    guard !faceAnchors.isEmpty,
        let camera = session.currentFrame?.camera,
        let targetView = SomeUIView() else { return }

    // Calculate face points to project to screen

    let projectionMatrix = camera.projectionMatrix(for: .portrait, viewportSize: targetView.bounds.size, zNear: 0.001, zFar: 1000)  // A transform matrix appropriate for rendering 3D content to match the image captured by the camera
    let viewMatrix = camera.viewMatrix(for: .portrait)        // Returns a transform matrix for converting from world space to camera space.

    let projectionViewMatrix = simd_mul(projectionMatrix, viewMatrix)

    for faceAnchor in faceAnchors  {

        let modelMatrix = faceAnchor.transform                  //  Describes the face’s current position and orientation in world coordinates; that is, in a coordinate space relative to that specified by the worldAlignment property of the session configuration. Use this transform matrix to position virtual content you want to “attach” to the face in your AR scene.
        let mvpMatrix = simd_mul(projectionViewMatrix, modelMatrix)

        // Calculate points

        let points: [CGPoint] = faceAnchor.geometry.vertices.flatMap({ (vertex) -> CGPoint? in

            let vertex4 = vector_float4(vertex.x, vertex.y, vertex.z, 1)

            let normalizedImageCoordinates = simd_mul(mvpMatrix, vertex4)

            return CGPoint(x: CGFloat(normalizedImageCoordinates.x ),
                           y: CGFloat(normalizedImageCoordinates.y ))
        })

    }

'项目'ARFaceAnchor指向屏幕

1 个答案: