Hololens字体相机

Question

我正在使用hololens，而我正试图获取前置摄像头的图像。我唯一需要的是拍摄相机每帧的图像并将其转换为字节数组。

Answer 1

1. 遵循文档中的Unity示例。它有一个写得很好的例子： https://docs.unity3d.com/Manual/windowsholographic-photocapture.html

从以上链接的统一文档中复制：

using UnityEngine;
using System.Collections;
using System.Linq;
using UnityEngine.XR.WSA.WebCam;

public class PhotoCaptureExample : MonoBehaviour {
PhotoCapture photoCaptureObject = null;
Texture2D targetTexture = null;

// Use this for initialization
void Start() {
    Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
    targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);

    // Create a PhotoCapture object
    PhotoCapture.CreateAsync(false, delegate (PhotoCapture captureObject) {
        photoCaptureObject = captureObject;
        CameraParameters cameraParameters = new CameraParameters();
        cameraParameters.hologramOpacity = 0.0f;
        cameraParameters.cameraResolutionWidth = cameraResolution.width;
        cameraParameters.cameraResolutionHeight = cameraResolution.height;
        cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;

        // Activate the camera
        photoCaptureObject.StartPhotoModeAsync(cameraParameters, delegate (PhotoCapture.PhotoCaptureResult result) {
            // Take a picture
            photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
        });
    });
}

void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame) {
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // Create a GameObject to which the texture can be applied
    GameObject quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
    Renderer quadRenderer = quad.GetComponent<Renderer>() as Renderer;
    quadRenderer.material = new Material(Shader.Find("Custom/Unlit/UnlitTexture"));

    quad.transform.parent = this.transform;
    quad.transform.localPosition = new Vector3(0.0f, 0.0f, 3.0f);

    quadRenderer.material.SetTexture("_MainTex", targetTexture);

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result) {
    // Shutdown the photo capture resource
    photoCaptureObject.Dispose();
    photoCaptureObject = null;
}
}

获取字节：替换以下方法：

    void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame) {
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // Create a GameObject to which the texture can be applied
    GameObject quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
    Renderer quadRenderer = quad.GetComponent<Renderer>() as Renderer;
    quadRenderer.material = new Material(Shader.Find("Custom/Unlit/UnlitTexture"));

    quad.transform.parent = this.transform;
    quad.transform.localPosition = new Vector3(0.0f, 0.0f, 3.0f);

    quadRenderer.material.SetTexture("_MainTex", targetTexture);

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

使用以下方法：

void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    List<byte> imageBufferList = new List<byte>();
    imageBufferList.Clear();
    photoCaptureFrame.CopyRawImageDataIntoBuffer(imageBufferList);
    var bytesArray = imageBufferList.ToArray();
}

1. 如果您现在正在使用Unity 2018.1或2018.2（我认为也是2017.4），那么它将无法正常工作。 Unity有公共跟踪器来解决它： https://issuetracker.unity3d.com/issues/windowsmr-failure-to-take-photo-capture-in-hololens
2. 我创建了一个解决方法，直到Unity修复了该错误： https://github.com/MSAlshair/HoloLensMediaCapture

不使用PhotoCapture统一解决方法的基本示例：上面的链接中有更多详细信息

您必须添加#if WINDOWS_UWP才能使用MediaCapture：理想情况下，您希望统一使用PhotoCapture来避免这种情况，但是在Unity解决此问题之前，您可以使用类似的方法。

#if WINDOWS_UWP
        public async System.Threading.Tasks.Task<byte[]> GetPhotoAsync()
        {
            //Get available devices info
            var devices = await Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(
                Windows.Devices.Enumeration.DeviceClass.VideoCapture);
            var numberOfDevices = devices.Count;

            byte[] photoBytes = null;

            //Check if the device has camera
            if (devices.Count > 0)
            {
                Windows.Media.Capture.MediaCapture mediaCapture = new Windows.Media.Capture.MediaCapture();
                await mediaCapture.InitializeAsync();

                //Get Highest available resolution
                var highestResolution = mediaCapture.VideoDeviceController.GetAvailableMediaStreamProperties(
                    Windows.Media.Capture.MediaStreamType.Photo).
                    Select(item => item as Windows.Media.MediaProperties.ImageEncodingProperties).
                    Where(item => item != null).
                    OrderByDescending(Resolution => Resolution.Height * Resolution.Width).
                    ToList().First();

                using (var photoRandomAccessStream = new Windows.Storage.Streams.InMemoryRandomAccessStream())
                {
                    await mediaCapture.CapturePhotoToStreamAsync(highestResolution, photoRandomAccessStream);

                    //Covnert stream to byte array
                    photoBytes = await ConvertFromInMemoryRandomAccessStreamToByteArrayAsync(photoRandomAccessStream);
                }
            }
            else
            {
                System.Diagnostics.Debug.WriteLine("No camera device detected!");
            }

            return photoBytes;
        }

        public static async System.Threading.Tasks.Task<byte[]> ConvertFromInMemoryRandomAccessStreamToByteArrayAsync(
            Windows.Storage.Streams.InMemoryRandomAccessStream inMemoryRandomAccessStream)
        {
            using (var dataReader = new Windows.Storage.Streams.DataReader(inMemoryRandomAccessStream.GetInputStreamAt(0)))
            {
                var bytes = new byte[inMemoryRandomAccessStream.Size];
                await dataReader.LoadAsync((uint)inMemoryRandomAccessStream.Size);
                dataReader.ReadBytes(bytes);

                return bytes;
            }
        }
#endif

不要忘记为清单添加功能：

1. WebCam
2. 麦克风：我不确定是否需要麦克风，因为我们只是在拍照还是不拍照，但是我还是添加了它。
3. 如果要保存到图片库

Answer 2

在播放器设置中，启用对相机的访问权限，然后重试。