Finding path obstacles in a 2D image

Question

what approach would you recommend for finding obstacles in a 2D image?

Here are some key points I came up with till now:

I doubt I can use object recognition based on "database of obstacles" search, since I don't know what might the obstruction look like. I assume color recognition might be problematic if the path does not differ a lot from the object itself.

Possibly, adding one more camera and computing a 3D image (like a Kinect does) would work, but that would not run as smooth as I require.

To illustrate the problem; robot can ride either left or right side of the pavement. In the following picture, left side is the correct choice:

Answer 1

If you know what the path looks like, this is largely a classification problem. Acquire a bunch of images of path at different distances, illumination, etc. and manually label the ground in each image. Use this labeled data to train a classifier that classifies each pixel as either "road" or "not road." Depending upon the texture of the road, this could be as simple as classifying each pixels' RGB (or HSV) values or using OpenCv's built-in histogram back-projection (i.e. cv::CalcBackProjectPatch()).

I suggest beginning with manual thresholds, moving to histogram-based matching, and only using a full-fledged machine learning classifier (such as a Naive Bayes Classifier or a SVM) if the simpler techniques fail. Once the entire image is classified, all pixels that are identified as "not road" are obstacles. By classifying the road instead of the obstacles, we completely avoided building a "database of objects".

---

Somewhat out of the scope of the question, the easiest solution is to add additional sensors ("throw more hardware at the problem!") and directly measure the three-dimensional position of obstacles. In order of preference:

1. Microsoft Kinect: Cheap, easy, and effective. Due to ambient IR light, it only works indoors.
2. Scanning Laser Rangefinder: Extremely accurate, easy to setup, and works outside. Also very expensive (~$1200-10,000 depending upon maximum range and sample rate).
3. Stereo Camera: Not as good as a Kinect, but it works outside. If you cannot afford a pre-made stereo camera (~$1800), you can make a decent custom stereo camera using USB webcams.

Note that professional stereo vision cameras can be very fast by using custom hardware (Stereo On-Chip, STOC). Software-based stereo is also reasonably fast (10-20 Hz) on a modern computer.