Explaining the AdaBoost Algorithms to non-technical people

Question

I\'ve been trying to understand the AdaBoost algorithm without much success. I\'m struggling with understanding the Viola Jones paper on Face Detection as an example.

Ca

Answer 1

This is understandable. Most of the papers you can find on Internet retell Viola-Jones and Freund-Shapire papers which are foundation of AdaBoost applied for face recognition in OpenCV. And they mostly consist of difficult formulas and algorithms from several mathematical areas combined. Here is what can help you (short enough) -

1 - It is used in object and, mostly, in face detection-recognition.The most popular and quite good C++ library is OpenCV from Intel originally. I take the part of Face detection in OpenCV, as an example.

2 - First, a cascade of boosted classifiers working with sample rectangles ("features") is trained on sample of images with faces (called positive) and without faces (negative).

From some Googled paper:

"· Boosting refers to a general and provably effective method of producing a very accurate classifier by combining rough and moderately inaccurate rules of thumb.

· It is based on the observation that finding many rough rules of thumb can be a lot easier than finding a single, highly accurate classifier.

· To begin, we define an algorithm for finding the rules of thumb, which we call a weak learner.

· The boosting algorithm repeatedly calls this weak learner, each time feeding it a different distribution over the training data (in AdaBoost).

· Each call generates a weak classifier and we must combine all of these into a single classifier that, hopefully, is much more accurate than any one of the rules."

During this process the images are scanned to determine the distinctive areas corresponding to certain part of every face. The complex calculation-hypothesis based algorithms are applied (which are not so difficult to understand once you get the main idea).

This can take a week and the output is an XML file which contains the learned information on how to quickly detect the human face, say, in frontal position on any picture (it can be any object in other case).

3 - After that you supply this file to OpenCV face detection program which runs quite fast with up to 99% positive rate (depending on conditions). As was mentioned here, the scanning speed can be increased greatly with technique known as "integral image".

And finally, these are helpful sources - Object Detection in OpenCV and Generic Object Detection using AdaBoost from University of California, 2008.

Answer 2

AdaBoost uses a number of training sample images (such as faces) to pick a number of good 'features'/'classifiers'. For face recognition a classifiers is typically just a rectangle of pixels that has a certain average color value and a relative size. AdaBoost will look at a number of classifiers and find out which one is the best predictor of a face based on the sample images. After it has chosen the best classifier it will continue to find another and another until some threshold is reached and those classifiers combined together will provide the end result.

This part you may not want to share with non-technical people :) but it is interesting anyway. There are several mathematical tricks which make AdaBoost fast for face recognition such as the ability to add up all the color values of an image and store them in a 2 dimensional array so that the value in any position will be the sum of all the pixels up and to the left of that position. This array can be used to very quickly calculate the average color value of any rectangle within the image by subtracting the value found in the top left corner from the value found in the bottom right corner and dividing by the number of pixels in the rectangle. Using this trick you can quickly scan over an entire image looking for rectangles of different relative sizes that match or are close to a particular color.

Hope this helps.

Answer 3

Adaboost is an algorithm that combines classifiers with poor performance, aka weak learners, into a bigger classifier with much higher performance.

How does it work? In a very simplified manner:

1. Train a weak learner.
2. Add it to the set of weak learners trained so far (with an optimal weight)
3. Increase the importance of samples that are still miss-classified.
4. Go to 1.

There is a broad and detailed theory behind the scenes, but the intuition is just that: let each "dumb" classifier focus on the mistakes the previous ones were not able to fix.

AdaBoost is one of the most used algorithms in the machine learning community. In particular, it is useful when you know how to create simple classifiers (possibly many different ones, using different features), and you want to combine them in an optimal way.

In Viola and Jones, each different type of weak-learner is associated to one of the 4 or 5 different Haar features you can have.