ImageMagick compare: Disregard white matches from the PSNR result

Question

I\'m using compare to diff two similar color PNG files. They get a PSNR value of ~27.

The images contain a lot of white areas that will always match bet

Answer 1

Like Mark Setchell, I do not fully understand what your real intention is. You do not provide any (links to) sample images which may help us to grasp what you are up to.

Preparation

That's why I created 4 sample images myself. Here is the first pair:

• The left one is a JPEG.
• The right one is a PNG.

Differences are minimal -- you'll hardly be able to spot them with the naked eye when reproduced here. Both images are 482x642 pixels:

 

Here comes the second pair. Again,

• the left one is a JPEG;
• the right one is a PNG.

These are both enlarged to 1002x1002 pixels by adding a white frame around the initial pair. In the following reproduction they appear smaller because of the different scaling inside this web page:

 

So the "important" parts of the second pair are what is contained in the black frames. The white frames on the outer bound is identical.

Comparison

Now lets compare both pairs. But I do not want to limit the comparisons to just the PSNR metric. I want to see all available metrics. We can list available metrics with this command:

compare -list metric
 AE
 Fuzz
 MAE
 MEPP
 MSE
 NCC
 PAE
 PHASH
 PSNR
 RMSE

My command to return the metrics for the first pair is this:

for metric in $(compare -list metric) ; do  \
   echo -n "Metric ${metric} :  " ;         \
   compare                                  \
      -metric ${metric}                     \
       http://i.stack.imgur.com/TEjAd.jpg   \
       http://i.stack.imgur.com/p8JsE.png   \
       null: ;                              \
   echo ;                                   \
done

Here is the result (slightly re-formatted) for the first pair ("important" part only):

Metric AE    :  123789
Metric Fuzz  :     948.522      (0.0144735)
Metric MAE   :     381.318      (0.00581854)
Metric MEPP  :       3.5399e+08 (0.000209349, 0.32549)
Metric MSE   :      13.7285     (0.000209483)
Metric NCC   :       0.998307
Metric PAE   :   21331          (0.32549)
Metric PHASH :       5.43771
Metric PSNR  :      36.7885
Metric RMSE  :     948.522      (0.0144735)

My command to return the metrics for the second pair is this:

for metric in $(compare -list metric) ; do  \
   echo -n "Metric ${metric} :  " ;         \
   compare                                  \
      -metric ${metric}                     \
       http://i.stack.imgur.com/gBruS.jpg   \
       http://i.stack.imgur.com/8NJeB.png   \
       null: ;                              \
   echo ;                                   \
done

Here is the result (slightly re-formatted) for the second pair ("watered down" differences by added massive white frame):

Metric AE    :  133609
Metric Fuzz  :     611.952       (0.00933779)
Metric MAE   :     143.849       (0.00219499)
Metric MEPP  :       4.33273e+08 (8.71895e-05, 0.341176)
Metric MSE   :       5.71428     (8.71944e-05)
Metric NCC   :       0.998137
Metric PAE   :   22359           (0.341176)
Metric PHASH :       0.360076
Metric PSNR  :      40.5951
Metric RMSE  :     611.952       (0.00933779)

Here are both results in a common table:

+==============+=======================================+=========================================+
| Metric Type  | Results for "important" image parts   | Results including "unimportant" frames  |
+==============+=======================================+=========================================+
| Metric AE    | 123789                                | 133609                                  |
| Metric Fuzz  |    948.522      (0.0144735)           |    611.952       (0.00933779)           |
| Metric MAE   |    381.318      (0.00581854)          |    143.849       (0.00219499)           |
| Metric MEPP  |      3.5399e+08 (0.000209349, 0.32549)|      4.33273e+08 (8.71895e-05, 0.341176)|
| Metric MSE   |     13.7285     (0.000209483)         |      5.71428     (8.71944e-05)          |
| Metric NCC   |      0.998307                         |      0.998137                           |
| Metric PAE   |  21331          (0.32549)             |  22359           (0.341176)             |
| Metric PHASH |      5.43771                          |      0.360076                           |
| Metric PSNR  |     36.7885                           |     40.5951                             |
| Metric RMSE  |    948.522      (0.0144735)           |    611.952       (0.00933779)           |
+==============+========================================+=========================================+

^{Note: comparing two identical images with the PSNR metric would result in an inf (infinitiv) value.}

Now draw your own conclusions...

Discussion

Understanding comparison metrics is not a straight forward affair.

Your own understanding of the PSNR seems to be a bit off, from how I interpret your 'Edit:'

To clarify:
Imagine this situation: you compare two images, they differ a bit. You then add a big white border to those images. You diff again and find that the resultant PSNR value has risen some amount.

Because a rising value for PSNR means that the two compared images have become a bit more identical to each other! (Of course, your intention to remove white (or otherwise colored) frames/borders around images before comparing them is still a reasonable approach. To see how to do that, look at the end of my answer...)

In order to get a better feeling for image comparison metrics, you should create a few simple "images" first. Then start experimenting with these.

Here is a suggestion how to create a series of mono-colored "patches", sized 100x100 pixels each:

for col in black white blue green red; do               \
  convert -size 100x100 xc:${col} xc-100px-${col}.png ; \
done

    

Experimental Task: Compare each of the 100x100 pixels patches against each other.
Questions:

• What is notable if you juxtapose the "black-white" metrics against the "black-blue" or the "red-green" metrics?
• Are some results unexpected by you? Why so?

Now do the same for patches with 200x200 pixels:

for col in black white blue green red; do               \
  convert -size 200x200 xc:${col} xc-200px-${col}.png ; \
done

Experimental Task: Compare each of the 200x200 pixels patches against each other.
Questions:

• Do the respective "color1-color2" metrics for the 200x200 images deviate from the same "color1-color2" metrics of the 100x100 images?
• Are there some metrics which return identical results to their respective counter-parts? Why so?

Now add a 50 pixels wide red frame around each of the 100x100 pixel patches. The resulting images will also be of size 200x200 pixels:

for img in xc-100px-*.png ; do \
   convert                     \
     ${img}                    \
    -mattecolor red            \
    -frame 50x50               \
     redframed-${img} ;        \
done

Experimental Task: Make up your own comparison pairs. (You know can also compare 200x200 pixel 'mono-color' patches against 200x200 pixels 'redframed' patches...)
Questions:

• Which are reasonable comparisons?
• Did you know that the -metric phash is the only one which allows you to compare images using different dimensions (width x height) ?

---

How to 'trim' a mono-colored frame around images

You can remove any 'frame' around an image that is composed of identically colored pixels. The image operator -trim will achieve this automatically for you. (It works for colors different than white too.)

convert reframed-xc-100px-blue.png -trim +repage output.png

identify redframed-xc-100px-blue.png  output.png
 redframed-xc-100px-blue.png PNG 200x200 200x200+0+0 8-bit sRGB 3c 322B 0.000u 0:00.000
               output.png[1] PNG 100x100 100x100+0+0 8-bit sRGB 2c 285B 0.000u 0:00.000