single person tracking from video sequence

Question

As a part of my thesis work, I need to build a program for human tracking from video or image sequence like the KTH or IXMAS dataset with the assumptions:

• Illu

Answer 1

Case 1 - If camera static

If the camera is static, it is really simple to track one person.

You can apply a method called background subtraction.

1. Here, for better results, you need a bare image from camera, with no persons in it. It is the background. ( It can also be done, even if you don't have this background image. But if you have it, better. I will tell at end what to do if no background image)

2. Now start capture from camera. Take first frame,convert both to grayscale, smooth both images to avoid noise.

3. Subtract background image from frame.

4. If the frame has no change wrt background image (ie no person), you get a black image ( Of course there will be some noise, we can remove it). If there is change, ie person walked into frame, you will get an image with person and background as black.

5. Now threshold the image for a suitable value.

6. Apply some erosion to remove small granular noise. Apply dilation after that.

7. Now find contours. Most probably there will be one contour,ie the person.

8. Find centroid or whatever you want for this person to track.

Now suppose you don't have a background image, you can find it using cvRunningAvg function. It finds running average of frames from your video which you use to track. But you can obviously understand, first method is better, if you get background image.

Here is the implementation of above method using cvRunningAvg.

Case 2 - Camera not static

Here background subtraction won't give good result, since you can't get a fixed background.

Then OpenCV come with a sample for people detection sample. Use it.

This is the file: peopledetect.cpp

I also recommend you to visit this SOF which deals with almost same problem: How can I detect and track people using OpenCV?

Answer 2

If you develop in .NET you can use the Aforge.NET framework.

http://www.aforgenet.com/

I was a regular visitor of the forums and I seem to remember there are plenty of people using it for tracking people.

I've also used the framework for other non-related purposes and can say I highly recommend it for its ease of use and powerful features.

Answer 3

The above method : a simple frame differencing followed by dilation and erosion would work, in case of a simple clean scene with just the motion of the person walking with absolutely no other motion or illumination changes. Also you are doing a detection every frame, as opposed to tracking. In this specific scenario, it might not be much more difficult to track either. Movement direction and speed : you can just run Lucas Kanade on the difference images.

At the core of it, what you need is a person detector followed by a tracker. Tracker can be either point based (Lucas Kanade or Horn and Schunck) or using Kalman filter or any of those kind of tracking for bounding boxes or blobs.

A lot of vision problems are ill-posed, some some amount of structure/constraints, helps to solve it considerably faster. Few questions to ask would be these :

1. Is the camera moving : No quite easy, Yes : Much harder, exactly what works depends on other conditions.
2. Is the scene constant except for the person
3. Is the person front facing / side-facing most of the time : Detect using viola jones or train one (adaboost with Haar or similar features) for side-facing face.
4. How spatially accurate do you need it to be, will a bounding box do it, or do you need a contour : Bounding box : just search (intelligently :)) in neighbourhood with SAD (sum of Abs Differences). Contour : Tougher, use contour based trackers.
5. Do you need the "tracklet" or only just the position of the person at each frame, temporal accuracy ?
6. What resolution are we speaking about here, since you need real time :
7. Is the scene sparse like the sequences or would it be cluttered ?
8. Is there any other motion in the sequence ?
9. Offline or Online ?

Answer 4

One possible solution is to use feature points tracking algorithm. Look at this book: Laganiere Robert - OpenCV 2 Computer Vision Application Programming Cookbook - 2011 p. 266

Full algorithm is already implemented in this book, using opencv.