OpenCV filtering ORB matches

Question

I am using the ORB feature detector to find matches between two images using this code:

    FeatureDetector detector = FeatureDetector.create(FeatureDetector.ORB);
    DescriptorExtractor descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);;
    DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

    // First photo
    Imgproc.cvtColor(img1, img1, Imgproc.COLOR_RGB2GRAY);
    Mat descriptors1 = new Mat();
    MatOfKeyPoint keypoints1 = new MatOfKeyPoint();

    detector.detect(img1, keypoints1);
    descriptor.compute(img1, keypoints1, descriptors1);

    // Second photo
    Imgproc.cvtColor(img2, img2, Imgproc.COLOR_RGB2GRAY);
    Mat descriptors2 = new Mat();
    MatOfKeyPoint keypoints2 = new MatOfKeyPoint();

    detector.detect(img2, keypoints2);
    descriptor.compute(img2, keypoints2, descriptors2);

    // Matching

    MatOfDMatch matches = new MatOfDMatch();
    MatOfDMatch filteredMatches = new MatOfDMatch();
    matcher.match(descriptors1, descriptors2, matches);

    // Linking
    Scalar RED = new Scalar(255,0,0);
    Scalar GREEN = new Scalar(0,255,0);

    List<DMatch> matchesList = matches.toList();
    Double max_dist = 0.0;
    Double min_dist = 100.0;

    for(int i = 0;i < matchesList.size(); i++){
        Double dist = (double) matchesList.get(i).distance;
        if (dist < min_dist)
            min_dist = dist;
        if ( dist > max_dist)
            max_dist = dist;
    }



    LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
    for(int i = 0;i < matchesList.size(); i++){
        if (matchesList.get(i).distance <= (1.5 * min_dist))
            good_matches.addLast(matchesList.get(i));
    }



    // Printing
    MatOfDMatch goodMatches = new MatOfDMatch();
    goodMatches.fromList(good_matches);

    System.out.println(matches.size() + " " + goodMatches.size());

    Mat outputImg = new Mat();
    MatOfByte drawnMatches = new MatOfByte();
    Features2d.drawMatches(img1, keypoints1, img2, keypoints2, goodMatches, outputImg, GREEN, RED, drawnMatches, Features2d.NOT_DRAW_SINGLE_POINTS);

    Highgui.imwrite("matches.png", outputImg);

My problem is that I can't find a way to filter the matches so that they only match when they have similar positions in the photos. I always get multiple matches for one keypoint even if they are very far away position-wise.

Is there a way to filter them better?

Answer 1

To get better matching results you should include these filtering methods in the given order.

1. Perform matching in two directions i.e for each point in first image find the best match in second image and vice versa .

2. Perform ratio test(ratio test of euclidean distances) between matches to eliminate ambiguous matches .

3. Perform RANSAC test: it's a model fitting algorithm which finds the best data which fits the model and removes outliers.
4. Perform homography: it is an image projection algorithm.

You can get all the details of above methods in chapter 9 of Computer vision application programming cookbook. It also has sample code for implementing these filtering techniques. It is very easy to understand. (Note: The code in this book is in C++ but once you understand, it can be easily implemented in JAVA too)

Answer 2

After reading Rober Langaniere book. I came to know there is a way. It is to remove matches with the further distances. In java, it is as following

Collections.sort(bestMatches,new Comparator<DMatch>() {
        @Override
        public int compare(DMatch o1, DMatch o2) {
            if(o1.distance<o2.distance)
                return -1;
            if(o1.distance>o2.distance)
                return 1;
            return 0;
        }
    });
    if(bestMatches.size()>3){
        bestMatches = bestMatches.subList(0,3);
    }

Answer 3

Matching is done by taking the shortest Hamming distance between two descriptors. So you will always get a match between the detected features.

You should modify the threshold of your ORB detector. This way, you will reduce the likelihood of detecting features from the background (i.e. noise), so the majority of your detected features would come from the subject of interest.