How can I detect yawn using Open CV

Question

I am developing an application for iOS that needs to detect when the user yawns.

What I did is include Open CV and find faces using a haar cascade, and then find mouth i

Answer 1

As a general, Support Vector Machine (SVM) is used for facial expression recognition like anger,smile, surprise etc where still active development takes place. Googling give you a lot of papers on this topic, (even one of my class mate did this as his final year project). For that, at first you need to train the SVM and to do that, you need sample images of yawning and normal faces.

Yawning is almost similar to surprise, where mouth open on both cases. I recommend you to check out page 3 of below paper : Real Time Facial Expression Recognition in Video using Support Vector Machines (If you can't access the link, google by paper name)

The paper (even my classmate) used displacement vector of facial features. For this, you find some feature points on the face. For example, in the paper, they have used eye pupil,extreme points of lids, nose tip, extreme points of mouth region (lips) etc. Then they continuously track the location of the features and find euclidean distance between them. They are used to train the SVM.

Check out below two papers :

Feature Points Extraction from Faces

Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers

Look image below what i mean by feature points on face:

In your case, i think you are implementing it in iPhone in real time. So may be you can avoid feature points at eyes (although not an good idea, since when you yawn, eyes become small in size). But compared to it, feature points at lips shows more variations and predominant. So, implementing on lip alone may save time. (Well, it all depends on you).

Lip Segmentation : It is already discussed in SOF and check out this question : OpenCV Lip Segmentation

And finally, i am sure you can find a lot of details on googling, because it is an active development area, and a lot of papers are out there.

Another Option :

Another option in this region, which i have heard several times, is Active Appearance Model. But I don't know anything about it. Google it yourself.