ImageMagick compare: Disregard white matches from the PSNR result

Question

I\'m using compare to diff two similar color PNG files. They get a PSNR value of ~27.

The images contain a lot of white areas that will always match bet

Answer 1

Like Mark Setchell, I do not fully understand what your real intention is. You do not provide any (links to) sample images which may help us to grasp what you are up to.

Preparation

That's why I created 4 sample images myself. Here is the first pair:

• The left one is a JPEG.
• The right one is a PNG.

Differences are minimal -- you'll hardly be able to spot them with the naked eye when reproduced here. Both images are 482x642 pixels:

 

Here comes the second pair. Again,

• the left one is a JPEG;
• the right one is a PNG.

These are both enlarged to 1002x1002 pixels by adding a white frame around the initial pair. In the following reproduction they appear smaller because of the different scaling inside this web page:

 

So the "important" parts of the second pair are what is contained in the black frames. The white frames on the outer bound is identical.

Comparison

Now lets compare both pairs. But I do not want to limit the comparisons to just the PSNR metric. I want to see all available metrics. We can list available metrics with this command:

compare -list metric
 AE
 Fuzz
 MAE
 MEPP
 MSE
 NCC
 PAE
 PHASH
 PSNR
 RMSE

My command to return the metrics for the first pair is this:

for metric in $(compare -list metric) ; do  \
   echo -n "Metric ${metric} :  " ;         \
   compare                                  \
      -metric ${metric}                     \
       http://i.stack.imgur.com/TEjAd.jpg   \
       http://i.stack.imgur.com/p8JsE.png   \
       null: ;                              \
   echo ;                                   \
done

Here is the result (slightly re-formatted) for the first pair ("important" part only):

Metric AE    :  123789
Metric Fuzz  :     948.522      (0.0144735)
Metric MAE   :     381.318      (0.00581854)
Metric MEPP  :       3.5399e+08 (0.000209349, 0.32549)
Metric MSE   :      13.7285     (0.000209483)
Metric NCC   :       0.998307
Metric PAE   :   21331          (0.32549)
Metric PHASH :       5.43771
Metric PSNR  :      36.7885
Metric RMSE  :     948.522      (0.0144735)

My command to return the metrics for the second pair is this:

for metric in $(compare -list metric) ; do  \
   echo -n "Metric ${metric} :  " ;         \
   compare                                  \
      -metric ${metric}                     \
       http://i.stack.imgur.com/gBruS.jpg   \
       http://i.stack.imgur.com/8NJeB.png   \
       null: ;                              \
   echo ;                                   \
done

Here is the result (slightly re-formatted) for the second pair ("watered down" differences by added massive white frame):

Metric AE    :  133609
Metric Fuzz  :     611.952       (0.00933779)
Metric MAE   :     143.849       (0.00219499)
Metric MEPP  :       4.33273e+08 (8.71895e-05, 0.341176)
Metric MSE   :       5.71428     (8.71944e-05)
Metric NCC   :       0.998137
Metric PAE   :   22359           (0.341176)
Metric PHASH :       0.360076
Metric PSNR  :      40.5951
Metric RMSE  :     611.952       (0.00933779)

Here are both results in a common table:

+==============+=======================================+=========================================+
| Metric Type  | Results for "important" image parts   | Results including "unimportant" frames  |
+==============+=======================================+=========================================+
| Metric AE    | 123789                                | 133609                                  |
| Metric Fuzz  |    948.522      (0.0144735)           |    611.952       (0.00933779)           |
| Metric MAE   |    381.318      (0.00581854)          |    143.849       (0.00219499)           |
| Metric MEPP  |      3.5399e+08 (0.000209349, 0.32549)|      4.33273e+08 (8.71895e-05, 0.341176)|
| Metric MSE   |     13.7285     (0.000209483)         |      5.71428     (8.71944e-05)          |
| Metric NCC   |      0.998307                         |      0.998137                           |
| Metric PAE   |  21331          (0.32549)             |  22359           (0.341176)             |
| Metric PHASH |      5.43771                          |      0.360076                           |
| Metric PSNR  |     36.7885                           |     40.5951                             |
| Metric RMSE  |    948.522      (0.0144735)           |    611.952       (0.00933779)           |
+==============+========================================+=========================================+

^{Note: comparing two identical images with the PSNR metric would result in an inf (infinitiv) value.}

Now draw your own conclusions...

Discussion

Understanding comparison metrics is not a straight forward affair.

Your own understanding of the PSNR seems to be a bit off, from how I interpret your 'Edit:'

To clarify:
Imagine this situation: you compare two images, they differ a bit. You then add a big white border to those images. You diff again and find that the resultant PSNR value has risen some amount.

Because a rising value for PSNR means that the two compared images have become a bit more identical to each other! (Of course, your intention to remove white (or otherwise colored) frames/borders around images before comparing them is still a reasonable approach. To see how to do that, look at the end of my answer...)

In order to get a better feeling for image comparison metrics, you should create a few simple "images" first. Then start experimenting with these.

Here is a suggestion how to create a series of mono-colored "patches", sized 100x100 pixels each:

for col in black white blue green red; do               \
  convert -size 100x100 xc:${col} xc-100px-${col}.png ; \
done

    

Experimental Task: Compare each of the 100x100 pixels patches against each other.
Questions:

• What is notable if you juxtapose the "black-white" metrics against the "black-blue" or the "red-green" metrics?
• Are some results unexpected by you? Why so?

Now do the same for patches with 200x200 pixels:

for col in black white blue green red; do               \
  convert -size 200x200 xc:${col} xc-200px-${col}.png ; \
done

Experimental Task: Compare each of the 200x200 pixels patches against each other.
Questions:

• Do the respective "color1-color2" metrics for the 200x200 images deviate from the same "color1-color2" metrics of the 100x100 images?
• Are there some metrics which return identical results to their respective counter-parts? Why so?

Now add a 50 pixels wide red frame around each of the 100x100 pixel patches. The resulting images will also be of size 200x200 pixels:

for img in xc-100px-*.png ; do \
   convert                     \
     ${img}                    \
    -mattecolor red            \
    -frame 50x50               \
     redframed-${img} ;        \
done

Experimental Task: Make up your own comparison pairs. (You know can also compare 200x200 pixel 'mono-color' patches against 200x200 pixels 'redframed' patches...)
Questions:

• Which are reasonable comparisons?
• Did you know that the -metric phash is the only one which allows you to compare images using different dimensions (width x height) ?

---

How to 'trim' a mono-colored frame around images

You can remove any 'frame' around an image that is composed of identically colored pixels. The image operator -trim will achieve this automatically for you. (It works for colors different than white too.)

convert reframed-xc-100px-blue.png -trim +repage output.png

identify redframed-xc-100px-blue.png  output.png
 redframed-xc-100px-blue.png PNG 200x200 200x200+0+0 8-bit sRGB 3c 322B 0.000u 0:00.000
               output.png[1] PNG 100x100 100x100+0+0 8-bit sRGB 2c 285B 0.000u 0:00.000

 

Answer 2

Updated

Ok, how about we make a mask of the areas you want to be compared first? So, if you want to ignore areas where both images are white, you can do this:

convert a.png b.png                   \
       -colorspace gray               \
       -compose multiply -composite   \
       -threshold 65534               \
       -negate PNG8:mask.png

Then, when you do your comparison, mask the images beforehand:

convert \( a.png mask.png -compose copy-opacity -composite \) \
        \( b.png mask.png -compose copy-opacity -composite \) \
        -metric PSNR -compare diff.png

or with compare like this in bash:

compare -metric PSNR \
    <(convert a.png mask.png -compose copy-opacity -composite PNG:-) \
    <(convert b.png mask.png -compose copy-opacity -composite PNG:-) \
    diff.png

I still don't see what you are trying to do, and I would still like to see your images and what result you expect... however, I have taken your advice and made two similar images from this page with the two s letters from the word missing blanked out.

a.png

b.png

So, if I now compare them, I get this:

convert a.png b.png -metric PSNR -compare -format "%[distortion]" info:
33.4539

and this image:

or if I do as I said, I get this:

convert -fill black \( a.png +opaque white \) \( b.png +opaque white \) -metric PSNR -compare -format "%[distortion]" info:
7.25418

and this image:

But I am still none the wiser about what you want because I have done all the work and you have not answered me.

Original Answer

It would be better if you could post your images so we can see what you mean, but try using convert like this to do the comparison:

convert a.png b.png -metric PSNR -compare diff.png

then you can add in pre-processing like this to make all white areas black before comparing. You can also add -fuzz 10% to catch near-white colours.

convert -fill black 
   \( a.png +opaque white \) \
   \( b.png +opaque white \) \
   -metric PSNR -compare diff.png