Some problems on image stitching - homography?

Question

I\'m working on a project of image stitching using OpenCV 2.3.1 on Visual Studio 2010.

I\'m currently having 2 problems.

(My reputation is not over 10 so I can o

Answer 1

Problem 1: I can't be very sure, but the problem with the stitching does seem to be due to the camera translation between the 2 pictures. With only a global homography transform, there's no way you can overlay the 2 images perfectly. Homography only suffices in the following 2 cases:

1. Camera undergoes a pure rotation (no translation)
2. Camera undergoes general motion, but the scene is planar

That said, your scene is fairly planar (objects are fairly far away compared to the translation of the camera) if not for the bottle. So an approximation by homography may still be sufficient. You just need to blend the images properly. To do so, you first need to find a place to "cut" the images where there is minimum difference between the 2 images, and apply (e.g. laplacian) blending. For your problem of cameras mounted on top of the car, this approximation may still be reasonable, so you may still be able to use a homography model.

If homography with proper blending is not sufficient, you may need to look at either 3D reconstruction techniques, or other methods that "relaxes" the homography requirement. There's a couple of papers in the literature that deals with parallax during mosaicking. These are however significantly more complex than the basic homography stitching though.

Problem 2: Yes, that can be done, as long you are very sure where the overlap is. However, you need to make sure that this overlapping region is not too small, or else the homography that you compute may be skewed. The problem with your office dataset appears to be due to camera translation, as explained before.

Lastly, you might want to adjust your SURF feature detection/matching parameters a bit. The feature points seem to be slightly on the low side.