how to perform stable eye corner detection?

Question

For those who find it too long, just read the bold lines.

My project of gaze estimation based screen cursor moving HCI is now dependent on one last thin

Answer 1

I think there is an easy way to help!

It looks as though you are considering each eye in isolation. What I suggest you do is to combine your data for both eyes, and also use facial geometry. I will illustrate my suggestions with a picture that some people may recognise (it is not really the best example, as its a painting, and her face is a bit off centre, but it is certainly the funniest..)

It seems you have relible estimates for the pupil position for both eyes, and providing the face is looking fairly straight on at the camera (facial rotations perpendicular to the screen will be ok using this method), we know that the corners of the eyes (from now on just 'corners') will lie on (or near to) the line that passes through the pupils of both eyes (red dotted line).

We know the distance between the pupils, a, and we know that the ratio between this distance, and the distance across one eye (corner to corner), b, is fixed for an individual, and will not change much across the adult population (may differ between sexes).

ie. a / b = constant.

Therefore we can deduce b, independent of the subjects distance from the camera, knowing only a.

Using this information we can construct threshold boxes for each eye corner (dotted boxes, in detail, labelled 1, 2, 3, 4). Each box is b by c (eye height, again determinable through the same fixed ratio principle) and lies parrallel to the pupil axis. The centre edge of each box is pinned to the centre of the pupil, and moves with it. We know each corner will always be in its very own threshold box!

Now, of course the trouble is the pupils move about, and so do our threshold boxes... but we've massively narrowed down the field this way, because we can confidently discard ALL estimate eye positions (from Harris or GFTT or anything) falling outside of these boxes (provided we are confident about our pupil detection).

• If we have high confidence in just one corner position we can extrapolate and deduce all the other corner postions just from geometry! (for both eyes!).

• If there is doubt between multiple corner positions we can use knowledge of other corners (from either eye) to resolve it probabilistically linking their positions, making a best guess. ie. do any pair of estimates (within their boxes of course) lie b apart and parallel to the pupil axis.

• If you can get general 'eye' positions that do not move when the pupil moves around (or in fact any facial feature on the same plane), this is massively useful and allows you to determine the corners positions geometrically.

I hope this might help you find the elusive d (pupil displacement from center of eye).