How to do Joint tracking in Kinect with a scaled Image

Question

I trying to do some Joint Tracking with kinect (just put a ellipse inside my right hand) everything works fine for a default 640x480 Image, i based myself in this channel9 video. My code, updated to use the new CoordinateMapper classe is here

 ...
CoordinateMapper cm = new CoordinateMapper(this.KinectSensorManager.KinectSensor);
ColorImagePoint handColorPoint = cm.MapSkeletonPointToColorPoint(atualSkeleton.Joints[JointType.HandRight].Position, ColorImageFormat.RgbResolution640x480Fps30);

Canvas.SetLeft(elipseHead, (handColorPoint.X) - (elipseHead.Width / 2)); // center of the ellipse in center of the joint
Canvas.SetTop(elipseHead, (handColorPoint.Y) - (elipseHead.Height / 2));

This works. The question is:

How to do joint tracking in a scaled image, 540x380 for example?

Answer 1

The solution for this is pretty simple, i fugured it out.

What a need to do is find some factor to apply to the position. This factor can be found takin the atual ColorImageFormat of the Kinect and dividing by the desired size, example:

Lets say i am working with the RgbResolution640x480Fps30 format and my Image (ColorViewer) have 220x240. So, lets find the factor for X:

double factorX = (640 / 220); // the factor is 2.90909090...

And the factor for y:

double factorY = (480/ 240); // the factor is 2...

Now, i adjust the position of the ellipse using this factor.

Canvas.SetLeft(elipseHead, (handColorPoint.X / (2.909090)) - (elipseHead.Width / 2));
Canvas.SetTop(elipseHead, (handColorPoint.Y / (2)) - (elipseHead.Height / 2));

Answer 2

I've not used the CoordinateMapper yet, and am not in front on my Kinect at the moment, so I'll toss out this first. I'll see about an update when I get working with the Kinect again.

The Coding4Fun Kinect Toolkit has a ScaleTo extension as part of the library. This adds the ability to take a joint and scale it to any display resolution.

The scaling function looks like this:

private static float Scale(int maxPixel, float maxSkeleton, float position)
{
    float value = ((((maxPixel / maxSkeleton) / 2) * position) + (maxPixel/2));
    if(value > maxPixel)
        return maxPixel;
    if(value < 0)
        return 0;
    return value;
}

maxPixel = the width or height, depending on which coordinate your scaling. maxSkeleton = set this to 1. position = the X or Y coordinate of the joint you want to scale.

If you were to just include the above function you could call it like so:

Canvas.SetLeft(e, Scale(640, 1, joint.Position.X));
Canvas.SetTop(e, Scale(480, 1, -joint.Position.Y));

... replacing your 640 & 480 with a different scale.

If you include the Coding4Fun Kinect Toolkit, instead of re-writing code, you could just call it like so:

scaledJoin = rawJoint.ScaleTo(640, 480);

... then plug in what you need.