Working with SSIM loss function in tensorflow for RGB images

邮差的信 提交于 2020-05-10 06:38:07

问题


I want to use SSIM metric as my loss function for the model I'm working on in tensorflow. SSIM should measure the similarity between my reconstructed output image of my denoising autoencoder and the input uncorrupted image (RGB).

As of what I understood, for using the SSIM metric in tensorflow, the images should be normalized to [0,1] or [0,255] and not [-1,1]. After converting my tensors to [0,1] and implementing SSIM as my loss function, the reconstructed image is black and white instead of a colorful RGB image.

tf.reduce_mean(tf.image.ssim(reconstructed, truth, 1.0))

My model is working fine with MSE (mean squared error), the reconstructed images are colorful (RGB).

using tf.losses.mean_squared_error(truth, reconstructed) the reconstructed image would be RGB image, while using SSIM would give me a one dimensional image.

Why using SSIM as loss function gives me different result than MSE (in terms of reconstructed image channels) in tensorflow?


回答1:


I was capable of solving the issue by changing the dynamic range of the images to 2.0, since I have images scaled between [-1, 1] by:

loss_rec = tf.reduce_mean(tf.image.ssim(truth, reconstructed, 2.0))

And since a better image quality is shown by a higher SSIM value, I had to minimize the negative of my loss function (SSIM) to optimize my model:

optimizer = tf.train.AdamOptimizer(learning_rate).minimize(-1 * loss_rec)




回答2:


SSIM is designed to only measure the difference between two luminance signals. The RGB images are converted to greyscale before measuring similarity. If that was fed back into the loss function, it wouldn't know if the image was losing color saturation because it wouldn't show up in the error metric. That's just a theory.




回答3:


The TensorFlow documentation says that no colorspace conversion is applied.

https://www.tensorflow.org/api_docs/python/tf/image/ssim

"Note: The true SSIM is only defined on grayscale. This function does not perform any colorspace transform. (If input is already YUV, then it will compute YUV SSIM average.)"



来源:https://stackoverflow.com/questions/52798540/working-with-ssim-loss-function-in-tensorflow-for-rgb-images

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!