OpenCV : Using a Trimap image

Question

I found this dog and cat image dataset: The Oxford-IIIT Pet Dataset. Each image has a pixel level foreground-background segmentation (trimap) image.

Searching the internet, I saw that trimap is an image with three colors (one for the background, one for the foreground and one for the not-classified region), but here the image is all black.

Is it a mistake or is it correct? But above all I want to know if and how you can use it to get, given a normal image, a new image with the cat or dog on a black background.

Thanks.

Answer 1

The trimaps look black because they only contain pixels values ranging from 0-2 on a scale of 0-255, where:

• 1 means "pet"
• 2 means "background"
• 3 means "border"

Look at the pixels in text form:

identify -verbose Abyssinian_1trimap.png  | more

Output

  Histogram:
     22938: (  1,  1,  1) #010101 gray(1)
    198766: (  2,  2,  2) #020202 gray(2)
     18296: (  3,  3,  3) #030303 gray(3)

If you contrast stretch the trimaps, you can see better. I am using the Abyssinian1 image here:

convert Abyssinian_1trimap.png -auto-level trimap.jpg

If you make all the 1 pixels in the trimap white and all the 2 pixels black and all the 3 pixels white and blend that with the actual photo using a darken blend, you will get what you want:

convert Abyssinian_1.jpg \( Abyssinian_1trimap.png -fill white -opaque "rgb(1,1,1)" -opaque "rgb(3,3,3)" -fill black -opaque "rgb(2,2,2)" \) -compose darken -composite pet.png

If you want the border as well as the pet, do this:

convert Abyssinian_1.jpg \( Abyssinian_1trimap.png -fill white -opaque "rgb(1,1,1)" -opaque "rgb(3,3,3)" -fill black -opaque "rgb(2,2,2)" \) -compose darken -composite pet.png

You can also experiment with blurring the mask to soften the edges:

convert Abyssinian_1.jpg \( Abyssinian_1trimap.png -fill white -opaque "rgb(1,1,1)" -fill black -opaque "rgb(3,3,3)" -opaque "rgb(2,2,2)" -blur 0x8  \) -compose darken -composite pet.png

Sorry, I did it with ImageMagick because I find that easier and it is installed on most Linux distros and available for macOS and Windows. The principles are the same for OpenCV.