scale and rotation Template matching

Question

I'm using the method of match template with CV_TM_CCORR_NORMED to compare two images ... I want to make to make this rotation and scale invariant .. any ideas?

I tried to use the same method on the fourier transform of the image and the template , but still the result after rotation is different

Answer 1

Template matching with matchTemplate is not good when your object is rotated or scaled in scene.

You should try openCV function from Features2D Framework. For example SIFT or SURF descriptors, and FLANN matcher. Also, you will need findHomography method.

Here is a good example of finding rotated object in scene.

Update:

In short, algorithm is this:

1. Finding keypoints of your object image 1.1. Extracting descriptors from those keypoints

2. Finding keypoints of your scene image 2.1 Extracting descriptors from keypoints

3. Match descriptors by matcher

4. Analyze your matches

There are different classes of FeatureDetectors, DescriptorExtractors, and DescriptorMatches, you may read about them and choose those, that fit good for your tasks.

• openCV FeatureDetector (steps 1 and 2 in algorithm above)
• openCV DescriptorExtractor ( steps 1.1 and 2.1 in algorithm above )
• openCV DescriptorMatcher ( step 3 in algorithm above )

Answer 2

Rotation invariant

For each key points:

1. Take area around key point.
2. Calculate orientation angle of this area with gradient or another method.
3. Rotate pattern and request area on this angle to 0.
4. Calculate descriptors for this rotated areas and match them.

Scale invariant

See BRISK method

Answer 3

There are easier ways of matching a template scale and rotationally invariant than going via feature detection and homographies (if you know its really only rotated and scales, but everything else is constant). For true object detection the above suggested keypoint based approaches work better.

If you know it's the same template and there is no perspective change involved, you take an image pyramid for scale-space detection, and match your templates on the different levels of that pyramid (via something simple, for example SSD or NCC). It will be cheap to find rough matches on higher (= lower resolution) levels of the pyramid. In fact, it will be so cheap, that you can also rotate your template roughly on the low resolution levels, and when you trace the template back down to the higher resolution levels, you use a more finely grained rotation stepping. That's a pretty standard template matching technique and works well in practice.