Facial Recognition with Kinect

Question

Lately I have been working on trying facial recognition with the Kinect, using the new Developer Toolkit (v1.5.1). The API for the FaceTracking tools can be found here: http://m

Answer 1

The picture you attached refers to the 2D model. GetProjected3DShape has nothing to do with the picture.

Use IFTResult.Get2DShapePoints to get 2D face points. If you are using the FaceTrackingBasics-WPF example, you have to write a C# wrapper for that method.

Answer 2

I working on a project like this one for my master's degree and I am computing distance using mahalanobis distance which is scale-invariant. Here is the formula: d(x,y)=sqrt(Pow((Xi-Yi),2)/Pow(Si,2)) ; i:1-->N, where Si is the standard deviation of the Xi and Yi over the sample set. Here is wikipedia link: http://en.wikipedia.org/wiki/Mahalanobis_distance

Answer 3

If you use a EnumIndexableCollection<FeaturePoint, PointF> so you can use a FaceTrackFrame's GetProjected3DShape() method. You use it like this:

  private byte[] colorImage;

  private ColorImageFormat colorImageFormat = ColorImageFormat.Undefined;

  private short[] depthImage;

  private DepthImageFormat depthImageFormat = DepthImageFormat.Undefined;

  KinectSensor Kinect = KinectSensor.KinectSensors[0];

  private Skeleton[] skeletonData;

  colorImageFrame = allFramesReadyEventArgs.OpenColorImageFrame();
  depthImageFrame = allFramesReadyEventArgs.OpenDepthImageFrame();
  skeletonFrame = allFramesReadyEventArgs.OpenSkeletonFrame();
  colorImageFrame.CopyPixelDataTo(this.colorImage);
  depthImageFrame.CopyPixelDataTo(this.depthImage);
  skeletonFrame.CopySkeletonDataTo(this.skeletonData);
  skeletonData = new Skeleton[skeletonFrame.SkeletonArrayLength];

  foreach(Skeleton skeletonOfInterest in skeletonData)
  {
       FaceTrackFrame frame = faceTracker.Track(
           colorImageFormat, colorImage, depthImageFormat, depthImage, skeletonOfInterest);
  }

  private EnumIndexableCollection<FeaturePoint, PointF> facePoints = frame.GetProjected3DShape();

Then you can use each of the points in your image. I would have a const double preferedDistance that you can multiply the current depth and x and y of the different points to find the preferred version of the x and y's and the depth by the formula

preferredDistance / currentDistance

Example:

        const double preferredDistance = 500.0;//this can be any number you want.

        double currentDistance = //however you are calculating the distance

        double whatToMultiply = preferredDistance / currentDistance;

        double x1 = this.facePoints[39].X;
        double y1 = this.facePoints[39].Y;
        double x2 = this.facePoints[8].X;
        double y2 = this.facePoints[8].Y;

        double result = whatToMultiply * //however you are calculating distance.

Then you can have a List<> of what the distances are to search. I would also suggest that you have a List<> of bool which coorispond to the distances to set to true if the result matches, so you can keep track of which bool is true/false.
Example:

        List<double> DistanceFromEyeToNose = new List<double>
        {
            1,
            2,
            3 //etc
        };


        List<bool> IsMatch = new List<bool>
        {
            false,
            false,
            false //etc
        };

Then search it by using a for loop.

        for (int i = 0; i < DistanceFromEyeToNose.Count; i++)
        {
            if (result == DistanceFromEyeToNose[i]) IsMatch[i] = true;
        } 

Hope this Helps!