Android - Face feature detection

Question

Currently I\'m working on an app for Android phones. We want to detect features of a face. The programm should be able to detect the positions of the eyes, the nose, the mouth a

Answer 1

Take a look at the new Android face API, which includes facial landmark detection. There is a tutorial here:

https://developers.google.com/vision/detect-faces-tutorial

Answer 2

it is a nice query. I guess if you get the feature points for eyes then we can calculate other points also by knowing the estimated distance of other points from eyes.

See this paper to know more about what I am trying to say: http://klucv2.googlecode.com/svn/trunk/docs/detection%20of%20facial%20feature%20points%20using%20anthropometric%20face%20model.pdf

I hope this helps.

Answer 3

OpenCV has a tutorial for this purpose, unfortunately is C++ only so you would have to convert it to Android.

You can also try FaceDetection API in Android, this is a simple example if you are detecting images from a drawable or sdcard images. Or the more recent Camera.Face API which works with the camera image.

If you want image from your camera at dynamic time than first read How to take picture from camera., but I would recommend you to check the official OpenCV Android samples and use them.

Updated:

Mad Hatter Example use the approach of Camera with SurfaceView. Its promisingly fast. Have a look at Mad Hatter.

The relevant code, in case the link goes down, is this:

public class FaceDetectionListener implements Camera.FaceDetectionListener {
    @Override
    public final void onFaceDetection(Face[] faces, Camera camera) {
        if (faces.length > 0) {
            for (Face face : faces) {
                if (face != null) {
                    // do something
                }
            }
        }
    }
}

Answer 4

I have found a good solution for face emotion detection provided by this Microsoft API. This API returns a JSON response and emotion graph. You can try this API for a good result.

Emotion API

Emotion Recognition Recognizes the emotions expressed by one or more people in an image, as well as returns a bounding box for the face. The emotions detected are happiness, sadness, surprise, anger, fear, contempt, and disgust or neutral.

• The supported input image formats includes JPEG, PNG, GIF(the first frame), BMP. Image file size should be no larger than 4MB.
• If a user has already called the Face API, they can submit the face rectangles as an optional input. Otherwise, Emotion API will first compute the rectangles.
• The detectable face size range is 36x36 to 4096x4096 pixels. Faces out of this range will not be detected.
• For each image, the maximum number of faces detected is 64 and the faces are ranked by face rectangle size in descending order. If no face is detected, an empty array will be returned.
• Some faces may not be detected due to technical challenges, e.g. very large face angles (head-pose), large occlusion. Frontal and near-frontal faces have the best results. -The emotions contempt and disgust are experimental.

https://www.microsoft.com/cognitive-services/en-us/emotion-api

Answer 5

I'm working on a similar project. I did some testing with the FaceDetection API and can tell you that it is not going to help you if you want to detect the eyes, nose, mouth and edges. This API only allow you to detect the eyes. It is useless if you want to implement face recognition because you need more features than just the eyes during the face detect part.

A comment on your first reply: you actually do need face detect. Finding features is part of face detection and getting these features is the first step in a face recognition app. With OpenCV you can use Haar-like features for getting these features (eyes, nose, mouth, etc.).

However I've found it somewhat complicated to use the openCV functions with a separate .cpp file. There is a thing called JNIEXPORT which allows you to edit an Android gallery image with OpenCV functions inside a .cpp file. OpenCV has a sample Haar-like feature detect .cpp file which can be used for face detection (and recognition as a second step with an other algorithm).

Are you developing on windows or linux? I'm using windows and haven't managed to use the tutorial you linked to set up OpenCV with it. However I do have a working windows OpenCV environment in Eclipse and got all samples from OpenCV 2.3.1 working. Maybe we can help each other out and share some information/results? please let me know.