Content-Based Image Retrieval and Precision-Recall graphs using Color Histograms in MATLAB

Question

So far, I have been able to plot Precision-Recall graphs for grayscale images in a CBIR system. However, I would like to know how to perform the same process for RGB images.

My code:

Inp1=rgb2gray(imread('D:\visionImages\c1\1.ppm'));
figure, imshow(Inp1), title('Input image 1');
num_bins = 32;
A = imhist(Inp1, num_bins);
srcFiles = dir('D:\visionImages\c1\*.ppm');  
B = zeros(num_bins, 30); 
ptr=1;
for i = 1 : length(srcFiles)
    filename = strcat('D:\visionImages\c1\',srcFiles(i).name);
    I = imread(filename);
    I=rgb2gray(I);
    B(:,ptr) = imhist(I, num_bins); 
    ptr=ptr+1;                                                      
end

% histogram intersection
a = size(A,2); b = size(B,2); 
K = zeros(a, b);
for i = 1:a
  Va = repmat(A(:,i),1,b);
  K(i,:) = 0.5*sum(Va + B - abs(Va - B));
end

num_images = 30;
sims=K
relevant_IDs = [1 2 3 4 5 6 7 8 9 10];
num_relevant_images = numel(relevant_IDs);
[sorted_sims, locs] = sort(sims, 'descend');
locations_final = arrayfun(@(x) find(locs == x, 1), relevant_IDs)
locations_sorted = sort(locations_final)
precision = (1:num_relevant_images) ./ locations_sorted;
recall = (1:num_relevant_images) / num_relevant_images;
plot(recall, precision, 'b.-');
xlabel('Recall');
ylabel('Precision');
title('Precision-Recall Graph');
axis([0 1 0 1.05]); 
grid;

Answer 1

The code that you have written is comparing the histogram between images, provided that they're grayscale. If you want to do this for RGB images, you need to determine how many bins you want per plane. Once you do this, for each RGB colour triplet that you have, you would determine a linear 1D index so that this would essentially act like a normal 1D histogram. Once you do this, you can use the above code in the same fashion as you have specified above. As such, let's create a function imcolourhist that takes in the image, and the total number of red, green and blue bins you want. Bear in mind that you can't specify 256 bins per dimension. Not only would this be too granular to have any discriminative power, but you would need 2^24 = 16777216 memory locations, and MATLAB will surely give you an out of memory error.

The general procedure would be to determine which bin each colour independently belongs to. Once you do this, you would then create a linear 1D index which is essentially a bin for a 1D histogram, and then you increment that value in that location. I'm going to use accumarray to calculate the histogram for me. Once we're done, this will essentially replace your imhist call and you would perform histogram intersection on this histogram that is output from imcolourhist instead.

function [out] = imcolourhist(im, num_red_bins, num_green_bins, num_blue_bins)

    im = double(im); %// To maintain precision

    %// Compute total number of bins
    total_bins = num_red_bins*num_green_bins*num_blue_bins;

    %// Figure out threshold between bins
    red_level = 256 / num_red_bins;
    green_level = 256 / num_green_bins;
    blue_level = 256 / num_blue_bins;

    %// Calculate which bins for each colour plane
    %// each pixel belongs to
    im_red_levels = floor(im(:,:,1) / red_level);
    im_green_levels = floor(im(:,:,2) / green_level);
    im_blue_levels = floor(im(:,:,3) / blue_level);

    %// Compute linear indices
    ind = im_blue_levels*num_red_bins*num_green_bins + im_green_levels*num_red_bins + im_red_levels;
    ind = ind(:); %// Make column vector for accumarray

    %// Determine 1D histogram - Ensure that every histogram
    %// generated has the same size of total_bins x 1
    out = accumarray(ind+1, 1, [total_bins 1]);
end

---

Take this code, copy and paste it into a new file, then save it as imcolourhist.m. Make sure you save this code in the same directory as where the above code you showed us is. Take note that in accumarray, I offset the linear indices by 1, as the linear indices that I generate will start from 0, but MATLAB starts indexing at 1. Now, all you have to do now is replace your imhist calls with imcolourhist. I would recommend you choose the bins per colour channel to be 8 for now (i.e. num_red_bins = num_green_bins = num_blue_bins = 8. You'll have to play around with this to get good results.

As such, you would change your code where you're computing the histogram for A as:

Inp1=imread('D:\visionImages\c1\1.ppm');  
num_red_bins = 8;
num_green_bins = 8;
num_blue_bins = 8;
num_bins = num_red_bins*num_green_bins*num_blue_bins;
A = imcolourhist(Inp1, num_red_bins, num_green_bins, num_blue_bins);

Note that I'm reading in the images as colour, so the rgb2gray call is removed. Similarly, for B, you would do:

B = zeros(num_bins, 30); 
ptr=1;
for i = 1 : length(srcFiles)
    filename = strcat('D:\visionImages\c1\',srcFiles(i).name);
    I = imread(filename);
    B(:,ptr) = imcolourhist(I, num_red_bins, num_green_bins, num_blue_bins); 
    ptr=ptr+1;                                                      
end

Note that I can't guarantee good results here. Because you're only using the colour histogram as a method for image retrieval, you could have a query image and a database image that may have the same colour distributions, but look completely different in terms of texture and composition. If both of these images have the same colour distribution, these would be considered as high similarity, even though they look nothing like each other.

---

Good luck!