Java OpenCV - Using knnMatch with findHomography shows duplicates

问题

I am new to OpenCV java and I have an android app that will match two images using ORB FeatureDetector and DescriptorExtractor. I use DescriptorMatcher BRUTEFORCE_HAMMING. All the time the matcher works but other times it shows duplicates of Keypoints. When Image on the Scene is too bright or too dark, it shows duplicate key points which is not what I wanted.

The image that accurately matches:

The image that is bad matches:

try {
 bmpObjToRecognize = bmpObjToRecognize.copy(Bitmap.Config.ARGB_8888, true);
 bmpScene = bmpScene.copy(Bitmap.Config.ARGB_8888, true);

 img1 = new Mat();
 img2 = new Mat();
 Utils.bitmapToMat(bmpObjToRecognize, img1);
 Utils.bitmapToMat(bmpScene, img2);
 Imgproc.cvtColor(img1, img1, Imgproc.COLOR_RGBA2GRAY);
 Imgproc.cvtColor(img2, img2, Imgproc.COLOR_RGBA2GRAY);
 Imgproc.equalizeHist(img1, img1);
 Imgproc.equalizeHist(img2, img2);
 detector = FeatureDetector.create(FeatureDetector.ORB);
 descExtractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
 matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

 keypoints1 = new MatOfKeyPoint();
 keypoints2 = new MatOfKeyPoint();
 descriptors = new Mat();
 dupDescriptors = new Mat();

 detector.detect(img1, keypoints1);
 Log.d("LOG!", "number of query Keypoints= " + keypoints1.size());
 detector.detect(img2, keypoints2);
 Log.d("LOG!", "number of dup Keypoints= " + keypoints2.size());
  // Descript keypoints1
  descExtractor.compute(img1, keypoints1, descriptors);
  descExtractor.compute(img2, keypoints2, dupDescriptors);
  // matching descriptors
  List<MatOfDMatch> knnMatches = new ArrayList<>();
  matcher.knnMatch(descriptors, dupDescriptors, knnMatches, DescriptorMatcher.BRUTEFORCE);
  goodMatches = new ArrayList<>();
  knnMatchesValue = knnMatches.size();
  Log.i("xxx", "xxx match count knnMatches = " + knnMatches.size());
  for (int i = 0; i < knnMatches.size(); i++) {
   if (knnMatches.get(i).rows() > 1) {
    DMatch[] matches = knnMatches.get(i).toArray();
    if (matches[0].distance < 0.89f * matches[1].distance) {
     goodMatches.add(matches[0]);
    }
   }
  }

  // get keypoint coordinates of good matches to find homography and remove outliers using ransac
  List<Point> pts1 = new ArrayList<>();
  List<Point> pts2 = new ArrayList<>();
  for (int i = 0; i < goodMatches.size(); i++) {
   Point destinationPoint = keypoints2.toList().get(goodMatches.get(i).trainIdx).pt;
   pts1.add(keypoints1.toList().get(goodMatches.get(i).queryIdx).pt);
   pts2.add(destinationPoint);
  }

  // convertion of data types - there is maybe a more beautiful way
  Mat outputMask = new Mat();
  MatOfPoint2f pts1Mat = new MatOfPoint2f();
  pts1Mat.fromList(pts1);
  MatOfPoint2f pts2Mat = new MatOfPoint2f();
  pts2Mat.fromList(pts2);

  // Find homography - here just used to perform match filtering with RANSAC, but could be used to e.g. stitch images
  // the smaller the allowed reprojection error (here 15), the more matches are filtered
  Mat Homog = Calib3d.findHomography(pts1Mat, pts2Mat, Calib3d.RANSAC, 15, outputMask, 2000, 0.995);

  // outputMask contains zeros and ones indicating which matches are filtered
  better_matches = new LinkedList<>();
  for (int i = 0; i < goodMatches.size(); i++) {
   if (outputMask.get(i, 0)[0] != 0.0) {
    better_matches.add(goodMatches.get(i));
   }
  }

  matches_final_mat = new MatOfDMatch();
  matches_final_mat.fromList(better_matches);

  imgOutputMat = new Mat();
  MatOfByte drawnMatches = new MatOfByte();
  Features2d.drawMatches(img1, keypoints1, img2, keypoints2, matches_final_mat, 
   imgOutputMat, GREEN, RED, drawnMatches, Features2d.NOT_DRAW_SINGLE_POINTS);
  bmp = Bitmap.createBitmap(imgOutputMat.cols(), imgOutputMat.rows(), Bitmap.Config.ARGB_8888);
  Imgproc.cvtColor(imgOutputMat, imgOutputMat, Imgproc.COLOR_BGR2RGB);
  Utils.matToBitmap(imgOutputMat, bmp);
  List<DMatch> betterMatchesList = matches_final_mat.toList();
  final int matchesFound = betterMatchesList.size();


} catch (Exception e) {
 e.printStackTrace();
}

Is there a part of the code that I am missing?

回答1:

TL;DR Use the class BFMatcher and its create method explicitly then your are able set the crosscheck flag to true. This will enable your wanted "vice versa check".

To cite the OpenCV documentation of knnMatch and its header:

Finds the k best matches for each descriptor from a query set.

knnMatch(InputArray queryDescriptors, InputArray trainDescriptors, ...)

So this means that it is possible that more than one of the "query descriptors" match to the same descriptor in the "training set". It just gives you the k best and if there are more query descriptors than training descriptors you will inevitably get duplicates. Especially, when you almost have no features and therefore descriptors in the training image/set (due to the lack of any texture e.g. your black input), that will be the case.

If you want to get rid of your duplicates, set the "crosscheck" flag of the BFMatcher to true. Otherwise (i.e. other matcher) you would need to go trough your matches "group" them by the respective training descriptors and remove all but the one with the smallest distance.

来源：https://stackoverflow.com/questions/58639964/java-opencv-using-knnmatch-with-findhomography-shows-duplicates