Kinect One和Matlab - 数据流显示

Question

当前版本的Matlab本身并不支持Kinect One。 Stackoverflow上有几篇帖子询问如何在Matlab环境中渲染特定的数据流（深度，骨架或rgb）。有没有人对如何在Matlab中记录/显示数据有任何建议或评论？

Answer 1

本文旨在解释如何在Matlab中呈现Kinect One的数据流。我总是喜欢单独录制数据而不是使用Matlab。对于那些正在挣扎的人来说，这是一个小帮助指南。

这篇文章的内容是什么： - Kinect One与Matlab（目前）不兼容。在这里，我发布了如何在Matlab中渲染骨架，RGB和深度图像的示例。这个想法是允许您在Matlab中使用数据执行操作。请注意，我只发布功能，您需要生成自己的显示。此外，您必须单独记录数据（在Kinect之外）。这些代码可能需要更改最新的SDK版本。你需要自己做一些腿部工作，但这些代码应该是一个很好的指标。

请注意，在我将文件更新到最新的SDK后的几周内，我将发布一个完整的界面（如下所示）。

此帖子将按以下方式格式化。对于每个流，我将在Matlab中发布要渲染的函数，以及如何从Kinect捕获数据的示例.cpp。代码有点乱，因为我已从现有项目中提取它。

如果您有任何疑问或改进意见，请发布。

<强>骨架：

.CPP代码段

 private void saveSkel(long timeStamp, Body body)
        {

            string filePath = yourPath + '\\' + timeStamp + ".txt";

            StreamWriter cooStream = new StreamWriter(filePath, false);

            IReadOnlyDictionary<JointType, Joint> joints = body.Joints;

            Dictionary<JointType, Point> jointPoints = new Dictionary<JointType, Point>();       

            foreach (JointType jointType in joints.Keys)
            {
                //Camera space points
                ColorSpacePoint depthSpacePoint = this.coordinateMapper.MapCameraPointToColorSpace(joints[jointType].Position);

                cooStream.WriteLine(joints[jointType].JointType +" " + joints[jointType].TrackingState + " " + joints[jointType].Position.X + " " + joints[jointType].Position.Y + " " + joints[jointType].Position.Z + " " + depthSpacePoint.X + " " + depthSpacePoint.Y);
            }
               //If we want to record both hand states
                if (handLassO == true)
                {
                    string wrtLineData = "LeftHand " + body.HandLeftState + " RightHand " + body.HandRightState;
                    cooStream.WriteLine(wrtLineData);

                }

            cooStream.Close();
        }

Matlab代码

功能1：

function handle = displayKinectSkeleton(ax, Skels)
%This function displays the Kinect skeleton
%
%Input:
%   ax - axis of the skeleton
%   Skels - skeleton file [format: 3*25 matrix (X;Y;Z)]
%
%Output:
%   handle - return the kinect handle 

%Index the joints
joints.SpineBase = 1; 
joints.SpineMid =2; 
joints.Neck =3; 
joints.Head =4; 
joints.ShoulderLeft =5; 
joints.ElbowLeft =6; 
joints.WristLeft =7; 
joints.HandLeft =8; 
joints.ShoulderRight =9; 
joints.ElbowRight =10; 
joints.WristRight =11; 
joints.HandRight =12; 
joints.HipLeft =13; 
joints.KneeLeft =14; 
joints.AnkleLeft =15; 
joints.FootLeft =16; 
joints.HipRight =17; 
joints.KneeRight =18; 
joints.AnkleRight =19; 
joints.FootRight =20; 
joints.SpineShoulder =21; 
joints.HandTipLeft =22;
joints.ThumbLeft =23;
joints.HandTipRight =24;
joints.ThumbRight =25;
joints.position_count = 25;

%Generate connections
skeleton_conn = [ ...
    %Torso
    joints.SpineBase, joints.SpineMid; ... %1
    joints.SpineMid,joints.SpineShoulder;... %2
    joints.SpineShoulder, joints.Neck;... %3
    joints.Neck,joints.Head;... %4

    %Left arm
    joints.SpineShoulder,joints.ShoulderLeft;... %5
    joints.ShoulderLeft, joints.ElbowLeft;... %6
    joints.ElbowLeft, joints.WristLeft;... %7
    joints.WristLeft, joints.HandLeft;... %8

    %Right arm
    joints.SpineShoulder, joints.ShoulderRight;... %9
    joints.ShoulderRight, joints.ElbowRight;...%10
    joints.ElbowRight, joints.WristRight;... %11
    joints.WristRight,joints.HandRight;... %12

    %Left leg
    joints.SpineBase, joints.HipLeft;... %13
    joints.HipLeft, joints.KneeLeft;... %14
    joints.KneeLeft,joints.AnkleLeft;... %15
    joints.AnkleLeft,joints.FootLeft;... %16

    %Right legs
    joints.SpineBase, joints.HipRight;... %17
    joints.HipRight, joints.KneeRight;... %18
    joints.KneeRight, joints.AnkleRight;... %19
    joints.AnkleRight, joints.FootRight; %20

    %Wrist
    joints.HandLeft, joints.HandTipLeft;... %21
    joints.WristLeft, joints.ThumbLeft;... %22

    %wrist
    joints.HandRight, joints.HandTipRight;... %23
    joints.WristRight, joints.ThumbRight;... %24

    ];

%Orignal XYZ points.I'be reshaped as I use the format elsewhere. 
xyz_shape = reshape(Skels, 3, 25)'; %I reshape as I use it for some different)
x = xyz_shape(:,1);
y = xyz_shape(:,2);
z = xyz_shape(:,3);

%Join the skeleton
xyz_step=[x y z];

%Plot the skeleton
handle = scatter3(ax, xyz_step(:,1), xyz_step(:,2), xyz_step(:,3), 'bo');

%Generate the limb connections
%Torso
torsoColor = 'r';
handle = displayLimbs(ax, xyz_step', skeleton_conn(1:4,:), torsoColor, 2.5);

%Left Arm
leftArmColor = 'g';
handle = displayLimbs(ax, xyz_step', skeleton_conn(5:7,:), leftArmColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(8:8,:), leftArmColor, 2.5);

%Right Arm
rightArmColor = 'b';
handle = displayLimbs(ax, xyz_step', skeleton_conn(9:11,:), rightArmColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(12:12,:), rightArmColor, 2.5);

%Legs
leftLegColor = 'y';
rightLegColor = 'm';
handle = displayLimbs(ax, xyz_step', skeleton_conn(13:14,:), leftLegColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(14:16,:), leftLegColor, 2.5);

handle = displayLimbs(ax, xyz_step', skeleton_conn(17:18,:), rightLegColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(18:20,:), rightLegColor, 2.5);

%wrist
wristColor = 'r';
handle = displayLimbs(ax, xyz_step', skeleton_conn(21:21,:), wristColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(22:22,:), wristColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(23:23,:), wristColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(24:24,:), wristColor, 2.5);
end

功能2：

function [ handle ] = displayLimbs(ax, xyz, jointLoc, color, width)
%This function joins specific joints together with the line function.
%
%Input:
%   ax - axis location
%   xyz - the xyz location for the time instance
%   jointLoc - joint index locations
%   color - color of the lines
%   width - width of the lines
%
%Output:
%   handle - Kinect handle
handle = line(xyz(1,jointLoc), xyz(2,jointLoc), xyz(3,jointLoc), 'Color', color, 'Parent', ax, 'LineWidth', width);
end

<强>深度：

.CPP代码段 *这是如何保存深度数据的示例。

 //Save the depth data

  string filePath = yourPath + '\\' + "depth" + frame.RelativeTime.ToString() + ".bin";

  using (FileStream streamDepth = new FileStream(filePath, FileMode.Create))
      {
         using (BinaryWriter depthWriter = new BinaryWriter(streamDepth))
       {
       depthWriter.Write(this.pixelsDepth);
         depthWriter.Close();
         }
         }

Matlab代码

功能1：
*使用上面的.cpp代码。读入并显示如下。

function [depthImg] = ReadKiectDepthIndivi(loc)
%This function reads in a Kinect depth image from a bin file
%
%Input:
%   loc - location of the bin file
%Output:
%   depthImg - depth image

%Read in the file
fid = fopen(loc); %read the file in
[A, count] = fread(fid, 'uint16=>uint16',2); %load and shift
fclose(fid); %close the stream

%Convert into structure
dims = [512 424]; %dims of the depth

depth = double(A); %convert to double for imaging

[depthImg.X, depthImg.Y] = meshgrid([1:dims(1)], [1:dims(2)]); %obtain x and y
depthImg.Z = reshape(depth, dims(1), dims(2))'; %generate depth
%temp.Z(temp.Z==0)=NaN; % noise clean up (if required)

end

使用以下命令渲染和显示：

surf(displayHandles.depthHandle, depthImg, depthImg, depthImg, 'EdgeColor','None', 'FaceAlpha', 0.5);

<强> RGB：

.CPP代码段 *为了速度和效率保存为bin文件。但是，取决于您的机器。您可能需要对记录速率进行下采样。<​​/ p>

string filePath = yourPath + '\\' + "image" + frame.RelativeTime.ToString() + ".bin";

using (FileStream streamRGB = new FileStream(filePath, FileMode.Create))
 {
  using (BinaryWriter rgbWriter = new BinaryWriter(streamRGB))
   {
   rgbWriter.Write(this.pixels);
   rgbWriter.Close();
 }

Matlab代码

功能1：

function [rgbImg] = ReadKinectRGBIndvid(loc)
%This function read's in a Kinect RGB image from a bin file
%
%Input:
%   loc - location of the bin file
%Ouput:
%   depthImg - depth image
%History:
%   Created by Dan 11/03/14 (based on JD version)

%Read in the image file
fid = fopen(loc);
[A, count] = fread(fid, 'uint8=>uint8');
fclose(fid);

dims = [1920 1080];
rgbImg(:,:,3) = reshape(A(1:4:end), dims(1), dims(2));
rgbImg(:,:,2) = reshape(A(2:4:end), dims(1), dims(2));
rgbImg(:,:,1) = reshape(A(3:4:end), dims(1), dims(2));

imageRGB = permute(rgbImg, [2 1 3]);

end

使用以下命令（或某种形式）渲染和显示：

RGB = imrotate(RGB ,-90);
RGB = flipdim(RGB,2);
image(RGB, 'Parent',displayHandles.rgbHandle);