single person tracking from video sequence

Question

As a part of my thesis work, I need to build a program for human tracking from video or image sequence like the KTH or IXMAS dataset with the assumptions:

• Illu

Answer 1

The above method : a simple frame differencing followed by dilation and erosion would work, in case of a simple clean scene with just the motion of the person walking with absolutely no other motion or illumination changes. Also you are doing a detection every frame, as opposed to tracking. In this specific scenario, it might not be much more difficult to track either. Movement direction and speed : you can just run Lucas Kanade on the difference images.

At the core of it, what you need is a person detector followed by a tracker. Tracker can be either point based (Lucas Kanade or Horn and Schunck) or using Kalman filter or any of those kind of tracking for bounding boxes or blobs.

A lot of vision problems are ill-posed, some some amount of structure/constraints, helps to solve it considerably faster. Few questions to ask would be these :

1. Is the camera moving : No quite easy, Yes : Much harder, exactly what works depends on other conditions.
2. Is the scene constant except for the person
3. Is the person front facing / side-facing most of the time : Detect using viola jones or train one (adaboost with Haar or similar features) for side-facing face.
4. How spatially accurate do you need it to be, will a bounding box do it, or do you need a contour : Bounding box : just search (intelligently :)) in neighbourhood with SAD (sum of Abs Differences). Contour : Tougher, use contour based trackers.
5. Do you need the "tracklet" or only just the position of the person at each frame, temporal accuracy ?
6. What resolution are we speaking about here, since you need real time :
7. Is the scene sparse like the sequences or would it be cluttered ?
8. Is there any other motion in the sequence ?
9. Offline or Online ?