问题
The current version of Matlab doesn't support Kinect One natively. There have been several posts here on Stackoverflow asking how to render the specific data streams (depth, skeleton or rgb) within the Matlab environment. Does anyone have any suggestions or comment as to how to record/display data within Matlab?
回答1:
This post is intended to explain how to render the data streams of Kinect One in Matlab. I’ve always preferred recording data separately than using Matlab. This is is a little help guide for those who are struggling.
What is this post about:- The Kinect One isn’t compatible with Matlab (currently). Here, I post examples of how to render the skeleton, RGB and depth image within Matlab. The idea, is to allow you to perform operations within Matlab with the data. Please note, I only post the functions, you need to generate your own display. Further, you must record the data separately (outside of the Kinect). These codes may need altering for the latest SDK release. You will need to do some of the leg work yourself but these codes should be a good indicator.
Please note, I will release a full interface (as shown below) in a couple of weeks after I’ve updated the files to the latest SDK.
This post will be formatted in the following way. For each stream I will post the functions in Matlab to render, and an example .cpp of how to capture the data from the Kinect. The code is a bit messy, as I’ve extracted it from an existing project.
If you do have any questions, or comments for improvement please post.
Skeleton:
.CPP Snippet
private void saveSkel(long timeStamp, Body body)
{
string filePath = yourPath + '\\' + timeStamp + ".txt";
StreamWriter cooStream = new StreamWriter(filePath, false);
IReadOnlyDictionary<JointType, Joint> joints = body.Joints;
Dictionary<JointType, Point> jointPoints = new Dictionary<JointType, Point>();
foreach (JointType jointType in joints.Keys)
{
//Camera space points
ColorSpacePoint depthSpacePoint = this.coordinateMapper.MapCameraPointToColorSpace(joints[jointType].Position);
cooStream.WriteLine(joints[jointType].JointType +" " + joints[jointType].TrackingState + " " + joints[jointType].Position.X + " " + joints[jointType].Position.Y + " " + joints[jointType].Position.Z + " " + depthSpacePoint.X + " " + depthSpacePoint.Y);
}
//If we want to record both hand states
if (handLassO == true)
{
string wrtLineData = "LeftHand " + body.HandLeftState + " RightHand " + body.HandRightState;
cooStream.WriteLine(wrtLineData);
}
cooStream.Close();
}
Matlab Code
Function 1:
function handle = displayKinectSkeleton(ax, Skels)
%This function displays the Kinect skeleton
%
%Input:
% ax - axis of the skeleton
% Skels - skeleton file [format: 3*25 matrix (X;Y;Z)]
%
%Output:
% handle - return the kinect handle
%Index the joints
joints.SpineBase = 1;
joints.SpineMid =2;
joints.Neck =3;
joints.Head =4;
joints.ShoulderLeft =5;
joints.ElbowLeft =6;
joints.WristLeft =7;
joints.HandLeft =8;
joints.ShoulderRight =9;
joints.ElbowRight =10;
joints.WristRight =11;
joints.HandRight =12;
joints.HipLeft =13;
joints.KneeLeft =14;
joints.AnkleLeft =15;
joints.FootLeft =16;
joints.HipRight =17;
joints.KneeRight =18;
joints.AnkleRight =19;
joints.FootRight =20;
joints.SpineShoulder =21;
joints.HandTipLeft =22;
joints.ThumbLeft =23;
joints.HandTipRight =24;
joints.ThumbRight =25;
joints.position_count = 25;
%Generate connections
skeleton_conn = [ ...
%Torso
joints.SpineBase, joints.SpineMid; ... %1
joints.SpineMid,joints.SpineShoulder;... %2
joints.SpineShoulder, joints.Neck;... %3
joints.Neck,joints.Head;... %4
%Left arm
joints.SpineShoulder,joints.ShoulderLeft;... %5
joints.ShoulderLeft, joints.ElbowLeft;... %6
joints.ElbowLeft, joints.WristLeft;... %7
joints.WristLeft, joints.HandLeft;... %8
%Right arm
joints.SpineShoulder, joints.ShoulderRight;... %9
joints.ShoulderRight, joints.ElbowRight;...%10
joints.ElbowRight, joints.WristRight;... %11
joints.WristRight,joints.HandRight;... %12
%Left leg
joints.SpineBase, joints.HipLeft;... %13
joints.HipLeft, joints.KneeLeft;... %14
joints.KneeLeft,joints.AnkleLeft;... %15
joints.AnkleLeft,joints.FootLeft;... %16
%Right legs
joints.SpineBase, joints.HipRight;... %17
joints.HipRight, joints.KneeRight;... %18
joints.KneeRight, joints.AnkleRight;... %19
joints.AnkleRight, joints.FootRight; %20
%Wrist
joints.HandLeft, joints.HandTipLeft;... %21
joints.WristLeft, joints.ThumbLeft;... %22
%wrist
joints.HandRight, joints.HandTipRight;... %23
joints.WristRight, joints.ThumbRight;... %24
];
%Orignal XYZ points.I'be reshaped as I use the format elsewhere.
xyz_shape = reshape(Skels, 3, 25)'; %I reshape as I use it for some different)
x = xyz_shape(:,1);
y = xyz_shape(:,2);
z = xyz_shape(:,3);
%Join the skeleton
xyz_step=[x y z];
%Plot the skeleton
handle = scatter3(ax, xyz_step(:,1), xyz_step(:,2), xyz_step(:,3), 'bo');
%Generate the limb connections
%Torso
torsoColor = 'r';
handle = displayLimbs(ax, xyz_step', skeleton_conn(1:4,:), torsoColor, 2.5);
%Left Arm
leftArmColor = 'g';
handle = displayLimbs(ax, xyz_step', skeleton_conn(5:7,:), leftArmColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(8:8,:), leftArmColor, 2.5);
%Right Arm
rightArmColor = 'b';
handle = displayLimbs(ax, xyz_step', skeleton_conn(9:11,:), rightArmColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(12:12,:), rightArmColor, 2.5);
%Legs
leftLegColor = 'y';
rightLegColor = 'm';
handle = displayLimbs(ax, xyz_step', skeleton_conn(13:14,:), leftLegColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(14:16,:), leftLegColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(17:18,:), rightLegColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(18:20,:), rightLegColor, 2.5);
%wrist
wristColor = 'r';
handle = displayLimbs(ax, xyz_step', skeleton_conn(21:21,:), wristColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(22:22,:), wristColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(23:23,:), wristColor, 2.5);
handle = displayLimbs(ax, xyz_step', skeleton_conn(24:24,:), wristColor, 2.5);
end
Function 2:
function [ handle ] = displayLimbs(ax, xyz, jointLoc, color, width)
%This function joins specific joints together with the line function.
%
%Input:
% ax - axis location
% xyz - the xyz location for the time instance
% jointLoc - joint index locations
% color - color of the lines
% width - width of the lines
%
%Output:
% handle - Kinect handle
handle = line(xyz(1,jointLoc), xyz(2,jointLoc), xyz(3,jointLoc), 'Color', color, 'Parent', ax, 'LineWidth', width);
end
Depth:
.CPP Snippet *This is an example of how to save the depth data.
//Save the depth data
string filePath = yourPath + '\\' + "depth" + frame.RelativeTime.ToString() + ".bin";
using (FileStream streamDepth = new FileStream(filePath, FileMode.Create))
{
using (BinaryWriter depthWriter = new BinaryWriter(streamDepth))
{
depthWriter.Write(this.pixelsDepth);
depthWriter.Close();
}
}
Matlab Code
Function 1:
*Using the .cpp code above. Read in and display as follows.
function [depthImg] = ReadKiectDepthIndivi(loc)
%This function reads in a Kinect depth image from a bin file
%
%Input:
% loc - location of the bin file
%Output:
% depthImg - depth image
%Read in the file
fid = fopen(loc); %read the file in
[A, count] = fread(fid, 'uint16=>uint16',2); %load and shift
fclose(fid); %close the stream
%Convert into structure
dims = [512 424]; %dims of the depth
depth = double(A); %convert to double for imaging
[depthImg.X, depthImg.Y] = meshgrid([1:dims(1)], [1:dims(2)]); %obtain x and y
depthImg.Z = reshape(depth, dims(1), dims(2))'; %generate depth
%temp.Z(temp.Z==0)=NaN; % noise clean up (if required)
end
Render and display using the following command:
surf(displayHandles.depthHandle, depthImg, depthImg, depthImg, 'EdgeColor','None', 'FaceAlpha', 0.5);
RGB:
.CPP Snippet *For speed and efficiency save as a bin file. However, depending on your machine. You may need to downsample the recording rate.
string filePath = yourPath + '\\' + "image" + frame.RelativeTime.ToString() + ".bin";
using (FileStream streamRGB = new FileStream(filePath, FileMode.Create))
{
using (BinaryWriter rgbWriter = new BinaryWriter(streamRGB))
{
rgbWriter.Write(this.pixels);
rgbWriter.Close();
}
Matlab Code
Function 1:
function [rgbImg] = ReadKinectRGBIndvid(loc)
%This function read's in a Kinect RGB image from a bin file
%
%Input:
% loc - location of the bin file
%Ouput:
% depthImg - depth image
%History:
% Created by Dan 11/03/14 (based on JD version)
%Read in the image file
fid = fopen(loc);
[A, count] = fread(fid, 'uint8=>uint8');
fclose(fid);
dims = [1920 1080];
rgbImg(:,:,3) = reshape(A(1:4:end), dims(1), dims(2));
rgbImg(:,:,2) = reshape(A(2:4:end), dims(1), dims(2));
rgbImg(:,:,1) = reshape(A(3:4:end), dims(1), dims(2));
imageRGB = permute(rgbImg, [2 1 3]);
end
Render and display using the following command (or some form):
RGB = imrotate(RGB ,-90);
RGB = flipdim(RGB,2);
image(RGB, 'Parent',displayHandles.rgbHandle);
来源：https://stackoverflow.com/questions/25700993/kinect-one-and-matlab-data-stream-display