Finding if two images are similar

Question

I\'m trying to find out it two images are similar with openCV using image matching. I\'m running the following code:

public static void match(String firstIma         


        

Answer 1

In layman terms, correlation determines the linear relationship between two variables/data-sets/etc.. The relationship maybe positively linear or negatively linear. Pearson's R is a common method for finding the correlation coefficient between two variables.

The result of Pearson's R lies between -1 and +1.

• -1 -> highly non linear to each other
• +1 -> highly linear to each other

It is a numerical way of stating how similar or dissimilar two variables are.

In this case, I am assuming the gray scale images to be the two variables. Prior to calculating it I am flattening the image into 1D arrays.

Pearson's R can be determined by using the scipy library (in Python).

from scipy.stats.stats import pearsonr

To get more information about pearsonr and its return type, check THIS LINK

So using python I computed the Pearson's R for all the three images, with each other.

This is what I got:

• For the first two images: (0.62908215058685268, 0.0)
• For the second and third image: (-0.34523397781005682, 0.0)
• For the first and third image: (-0.36356339857880066, 0.0)

Hence we can say that (in numerical/statistical terms) the first two images are somewhat similar (given the fact that one has a handbag and the other doesn't).

If you perform Pearson's R for two same images, you will obtain a result of 1

Answer 2

For checking the appropriateness of homography matrix between two perspectives of similar images, I wrote this function. You can choose what you prefer out of the possibilities, depending on which one you find more suitable:

 private static boolean check_homography(Mat homography_mat){
    /* Check 1. Compute the determinant of the homography, and see if it's too close 
     to zero for comfort*/
    if(!homography_mat.empty())
    {
        double Determinant = Core.determinant(homography_mat);
        if (Determinant > 0.1)
            return true;
        else 
            return false;
    }
    else 
        return false;

    /* Check 2. Compute its SVD, and verify that the ratio of the first-to-last 
     singular value is not too high (order of 1.0E7). */   
    Mat singularValues = new Mat();
    Core.SVDecomp(homography_mat, singularValues, new Mat(), new Mat(), Core.SVD_NO_UV);

    System.out.print("\n Printing the singular values of the homography");
    for (int i = 0; i < singularValues.rows(); i++){
        for ( int j = 0; j < singularValues.cols(); j++){
            System.out.print("\n Element at ( " + i + ", " + j + " ) is " + singularValues.get(i, j)[0]);
        }
    }
    double conditionNumber = singularValues.get(0, 0)[0] / singularValues.get(2, 0)[0];
    System.out.print("\n Condition number is : " + conditionNumber);

    if(conditionNumber < Math.pow(10, 7)){
        System.out.print("\n Homography matrix is non-singular");
        return true;
        }
    else{
        System.out.print("\n Homography matrix is singular (or very close)");
        return false;
        }
    /* Check 3. Check the compare absolute values at (0,0) and (0,1) with (1,1) and (1,0) 
     * respectively. If the two differences are close to 0, the homography matrix is 
     * good. (This just takes of rotation and not translation)
     * */
    if(Math.abs((Math.abs(homography_mat.get(0, 0)[0]) - Math.abs(homography_mat.get(1, 1)[0]))) <= 0.1){
        if(Math.abs((Math.abs(homography_mat.get(0, 1)[0]) - Math.abs(homography_mat.get(1, 0)[0]))) <= 0.1){
            System.out.print("\n The homography matrix is good");
            return true;
        }
    }
        else{
            System.out.print("\n The homography matrix is bad");
            return false;
        }
    return false;
    /*
     * Check 4: If the determinant of the top-left 2 by 2 matrix (rotation) > 0, transformation is orientation
     * preserving.
     * Else if the determinant is < 0, it is orientation reversing
     * 
     * */
     Determinant of the rotation mat
    double det = homography_mat.get(0, 0)[0] * homography_mat.get(1,1)[0] - homography_mat.get(0, 1)[0] * homography_mat.get(1, 0)[0];
    if (det < 0)
        return false;

    double N1 = Math.sqrt(homography_mat.get(0, 0)[0] * homography_mat.get(0, 0)[0] + homography_mat.get(1, 0)[0] * homography_mat.get(1, 0)[0]);
    if (N1 > 4 || N1 < 0.1)
        return false;

    double N2 = Math.sqrt(homography_mat.get(0, 1)[0] * homography_mat.get(0, 1)[0] + homography_mat.get(1, 1)[0] * homography_mat.get(1, 1)[0]);
    if (N2 > 4 || N2 < 0.1)
        return false;

    double N3 = Math.sqrt(homography_mat.get(2, 0)[0] * homography_mat.get(2, 0)[0] + homography_mat.get(2,1)[0] * homography_mat.get(2, 1)[0]);
    if (N3 < 0.002)
        return false;

    return true;

}

Note - I coded this for Java, OpenCV when using ORB. I personally (with experience I guess), can look at the Homography matrix and say more or less if its good or not and hence the Check 1. Hope it helps!!

EDIT Additionally, as @Micka mentioned, this is subject to your preference, you can also iterate along the two images pixel by pixel to find the similarity. The method I post here is used to check if the homography is good or not, however, it should be noted that you can get bad homography even when two images are similar depending on the images and the method(pre/post processing, descriptor, detector etc. )