Multi otsu(multi-thresholding) with openCV

Question

I am trying to carry out multi-thresholding with otsu. The method I am using currently is actually via maximising the between class variance, I have managed to get the same

Answer 1

@Antoni4 gives the best answer in my opinion and it's very straight forward to increase the number of levels.

This is for three-level thresholding:

#include "Shadow01-1.cuh"

void multiThresh(double &optimalThresh1, double &optimalThresh2, double &optimalThresh3, cv::Mat &imgHist, cv::Mat &src)
{
double W0K, W1K, W2K, W3K, M0, M1, M2, M3, currVarB, maxBetweenVar, M0K, M1K, M2K, M3K, MT;
unsigned char *histogram = (unsigned char*)(imgHist.data);

int N = src.rows*src.cols;
W0K = 0;
W1K = 0;
M0K = 0;
M1K = 0;
MT = 0;
maxBetweenVar = 0;

for (int k = 0; k <= 255; k++) {
    MT += k * (histogram[k] / (double) N);
}

for (int t1 = 0; t1 <= 255; t1++)
{
    W0K += histogram[t1] / (double) N; //Pi
    M0K += t1 * (histogram[t1] / (double) N); //i * Pi
    M0 = M0K / W0K; //(i * Pi)/Pi

    W1K = 0;
    M1K = 0;

    for (int t2 = t1 + 1; t2 <= 255; t2++)
    {
        W1K += histogram[t2] / (double) N; //Pi
        M1K += t2 * (histogram[t2] / (double) N); //i * Pi
        M1 = M1K / W1K; //(i * Pi)/Pi
        W2K = 1 - (W0K + W1K);
        M2K = MT - (M0K + M1K);

        if (W2K <= 0) break;

        M2 = M2K / W2K;

        W3K = 0;
        M3K = 0;

        for (int t3 = t2 + 1; t3 <= 255; t3++)
        {
            W2K += histogram[t3] / (double) N; //Pi
            M2K += t3 * (histogram[t3] / (double) N); // i*Pi
            M2 = M2K / W2K; //(i*Pi)/Pi
            W3K = 1 - (W1K + W2K);
            M3K = MT - (M1K + M2K);

            M3 = M3K / W3K;
            currVarB = W0K * (M0 - MT) * (M0 - MT) + W1K * (M1 - MT) * (M1 - MT) + W2K * (M2 - MT) * (M2 - MT) + W3K * (M3 - MT) * (M3 - MT);

            if (maxBetweenVar < currVarB)
            {
                maxBetweenVar = currVarB;
                optimalThresh1 = t1;
                optimalThresh2 = t2;
                optimalThresh3 = t3;
            }
        }
    }
}
}

Answer 2

To extend Otsu's thresholding method to multi-level thresholding the between class variance equation becomes:

Please check out Deng-Yuan Huang, Ta-Wei Lin, Wu-Chih Hu, Automatic Multilevel Thresholding Based on Two-Stage Otsu's Method with Cluster Determination by Valley Estimation, Int. Journal of Innovative Computing, 2011, 7:5631-5644 for more information.

http://www.ijicic.org/ijicic-10-05033.pdf

Here is my C# implementation of Otsu Multi for 2 thresholds:

/* Otsu (1979) - multi */

Tuple < int, int > otsuMulti(object sender, EventArgs e) {
    //image histogram
    int[] histogram = new int[256];

    //total number of pixels
    int N = 0;

    //accumulate image histogram and total number of pixels
    foreach(int intensity in image.Data) {
        if (intensity != 0) {
            histogram[intensity] += 1;
            N++;
        }
    }

    double W0K, W1K, W2K, M0, M1, M2, currVarB, optimalThresh1, optimalThresh2, maxBetweenVar, M0K, M1K, M2K, MT;

    optimalThresh1 = 0;
    optimalThresh2 = 0;

    W0K = 0;
    W1K = 0;

    M0K = 0;
    M1K = 0;

    MT = 0;
    maxBetweenVar = 0;
    for (int k = 0; k <= 255; k++) {
        MT += k * (histogram[k] / (double) N);
    }


    for (int t1 = 0; t1 <= 255; t1++) {
        W0K += histogram[t1] / (double) N; //Pi
        M0K += t1 * (histogram[t1] / (double) N); //i * Pi
        M0 = M0K / W0K; //(i * Pi)/Pi

        W1K = 0;
        M1K = 0;

        for (int t2 = t1 + 1; t2 <= 255; t2++) {
            W1K += histogram[t2] / (double) N; //Pi
            M1K += t2 * (histogram[t2] / (double) N); //i * Pi
            M1 = M1K / W1K; //(i * Pi)/Pi

            W2K = 1 - (W0K + W1K);
            M2K = MT - (M0K + M1K);

            if (W2K <= 0) break;

            M2 = M2K / W2K;

            currVarB = W0K * (M0 - MT) * (M0 - MT) + W1K * (M1 - MT) * (M1 - MT) + W2K * (M2 - MT) * (M2 - MT);

            if (maxBetweenVar < currVarB) {
                maxBetweenVar = currVarB;
                optimalThresh1 = t1;
                optimalThresh2 = t2;
            }
        }
    }

    return new Tuple(optimalThresh1, optimalThresh2);
}

And this is the result I got by thresholding an image scan of soil with the above code:

(T1 = 110, T2 = 147).

Otsu's original paper: "Nobuyuki Otsu, A Threshold Selection Method from Gray-Level Histogram, IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9:62-66" also briefly mentions the extension to Multithresholding.

https://engineering.purdue.edu/kak/computervision/ECE661.08/OTSU_paper.pdf

Hope this helps.

Answer 3

I've written an example on how otsu thresholding work in python before. You can see the source code here: https://github.com/subokita/Sandbox/blob/master/otsu.py

In the example there's 2 variants, otsu2() which is the optimised version, as seen on Wikipedia page, and otsu() which is more naive implementation based on the algorithm description itself.

If you are okay in reading python codes (in this case, they are pretty simple, almost pseudo code like), you might want to look at otsu() in the example and modify it. Porting it to C++ code is not hard either.

Answer 4

@Guilherme Silva

Your code has a BUG

You Must Replace:

    W3K = 0;
    M3K = 0;

with

    W2K = 0;
    M2K = 0;

and

        W3K = 1 - (W1K + W2K);
        M3K = MT - (M1K + M2K);

with

        W3K = 1 - (W0K + W1K + W2K);
        M3K = MT - (M0K + M1K + M2K);

;-) Regards

EDIT(1): [Toby Speight] I discovered this bug by applying the effect to the same picture at different resoultions(Sizes) and seeing that the output results were to much different from each others (Even changing resolution a little bit)

W3K and M3K must be the totals minus the Previous WKs and MKs. (I thought about this for Code-similarity with the one with one level less) At the moment due to my lacks of English I cannot explain Better How and Why

To be honest I'm still not 100% sure that this way is correct, even thought from my outputs I could tell that it gives better results. (Even with 1 Level more (5 shades of gray)) You could try yourself ;-) Sorry

My Outputs:

3 Thresholds 4 Thresholds

Answer 5

I found a useful piece of code in this thread. I was looking for a multi-level Otsu implementation for double/float images. So, I tried to generalize example for N-levels with double/float matrix as input. In my code below I am using armadillo library as dependency. But this code can be easily adapted for standard C++ arrays, just replace vec, uvec objects with single dimensional double and integer arrays, mat and umat with two-dimensional. Two other functions from armadillo used here are: vectorise and hist.

// Input parameters:
// map - input image (double matrix)
// mask - region of interest to be thresholded
// nBins - number of bins
// nLevels - number of Otsu thresholds

#include <armadillo>
#include <algorithm>
#include <vector>

mat OtsuFilterMulti(mat map, int nBins, int nLevels) {

    mat mapr;   // output thresholded image
    mapr = zeros<mat>(map.n_rows, map.n_cols);

    unsigned int numElem = 0;
    vec threshold = zeros<vec>(nLevels);
    vec q = zeros<vec>(nLevels + 1);
    vec mu = zeros<vec>(nLevels + 1);
    vec muk = zeros<vec>(nLevels + 1);
    uvec binv = zeros<uvec>(nLevels);

    if (nLevels <= 1) return mapr;

    numElem = map.n_rows*map.n_cols;


    uvec histogram = hist(vectorise(map), nBins);

    double maxval = map.max();
    double minval = map.min();
    double odelta = (maxval - abs(minval)) / nBins;     // distance between histogram bins

    vec oval = zeros<vec>(nBins);
    double mt = 0, variance = 0.0, bestVariance = 0.0;

    for (int ii = 0; ii < nBins; ii++) {
        oval(ii) = (double)odelta*ii + (double)odelta*0.5;  // centers of histogram bins
        mt += (double)ii*((double)histogram(ii)) / (double)numElem;
    }

    for (int ii = 0; ii < nLevels; ii++) {
        binv(ii) = ii;
    }

    double sq, smuk;
    int nComb;

    nComb = nCombinations(nBins,nLevels);
    std::vector<bool> v(nBins);
    std::fill(v.begin(), v.begin() + nLevels, true);

    umat ibin = zeros<umat>(nComb, nLevels); // indices from combinations will be stored here

    int cc = 0;
    int ci = 0;
    do {
        for (int i = 0; i < nBins; ++i) {
            if(ci==nLevels) ci=0;
                if (v[i]) {
                ibin(cc,ci) = i;
                ci++;
            }
        }
        cc++;
    } while (std::prev_permutation(v.begin(), v.end()));

    uvec lastIndex = zeros<uvec>(nLevels);

    // Perform operations on pre-calculated indices
    for (int ii = 0; ii < nComb; ii++) {
        for (int jj = 0; jj < nLevels; jj++) {
            smuk = 0;
            sq = 0;
            if (lastIndex(jj) != ibin(ii, jj) || ii == 0) {
                q(jj) += double(histogram(ibin(ii, jj))) / (double)numElem;
                muk(jj) += ibin(ii, jj)*(double(histogram(ibin(ii, jj)))) / (double)numElem;
                mu(jj) = muk(jj) / q(jj);
                q(jj + 1) = 0.0;
                muk(jj + 1) = 0.0;

                if (jj>0) {
                    for (int kk = 0; kk <= jj; kk++) {
                        sq += q(kk);
                        smuk += muk(kk);
                    }
                    q(jj + 1) = 1 - sq;
                    muk(jj + 1) = mt - smuk;
                    mu(jj + 1) = muk(jj + 1) / q(jj + 1);
                }
                if (jj>0 && jj<(nLevels - 1)) {
                    q(jj + 1) = 0.0;
                    muk(jj + 1) = 0.0;
                }

                lastIndex(jj) = ibin(ii, jj);
            }
        }

        variance = 0.0;
        for (int jj = 0; jj <= nLevels; jj++) {
            variance += q(jj)*(mu(jj) - mt)*(mu(jj) - mt);
        }    

        if (variance > bestVariance) {
            bestVariance = variance;
            for (int jj = 0; jj<nLevels; jj++) {
                threshold(jj) = oval(ibin(ii, jj));
            }
        }    
    }

    cout << "Optimized thresholds: ";
    for (int jj = 0; jj<nLevels; jj++) {
        cout << threshold(jj) << " ";
    }
    cout << endl;

    for (unsigned int jj = 0; jj<map.n_rows; jj++) {
        for (unsigned int kk = 0; kk<map.n_cols; kk++) {
            for (int ll = 0; ll<nLevels; ll++) {
                if (map(jj, kk) >= threshold(ll)) {
                    mapr(jj, kk) = ll+1;
                }
            }
        }
    }

    return mapr;
}


int nCombinations(int n, int r) {

    if (r>n) return 0;
    if (r*2 > n) r = n-r;
    if (r == 0) return 1;

    int ret = n;
    for( int i = 2; i <= r; ++i ) {
        ret *= (n-i+1);
        ret /= i;
    }
    return ret;
}

Answer 6

Here is a simple general approach for 'n' thresholds in python (>3.0) :

# developed by- SUJOY KUMAR GOSWAMI
# source paper- https://people.ece.cornell.edu/acharya/papers/mlt_thr_img.pdf

import cv2
import numpy as np
import math

img = cv2.imread('path-to-image')
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
a = 0
b = 255
n = 6 # number of thresholds (better choose even value)
k = 0.7 # free variable to take any positive value
T = [] # list which will contain 'n' thresholds

def sujoy(img, a, b):
    if a>b:
        s=-1
        m=-1
        return m,s

    img = np.array(img)
    t1 = (img>=a)
    t2 = (img<=b)
    X = np.multiply(t1,t2)
    Y = np.multiply(img,X)
    s = np.sum(X)
    m = np.sum(Y)/s
    return m,s

for i in range(int(n/2-1)):
    img = np.array(img)
    t1 = (img>=a)
    t2 = (img<=b)
    X = np.multiply(t1,t2)
    Y = np.multiply(img,X)
    mu = np.sum(Y)/np.sum(X)

    Z = Y - mu
    Z = np.multiply(Z,X)
    W = np.multiply(Z,Z)
    sigma = math.sqrt(np.sum(W)/np.sum(X))

    T1 = mu - k*sigma
    T2 = mu + k*sigma

    x, y = sujoy(img, a, T1)
    w, z = sujoy(img, T2, b)

    T.append(x)
    T.append(w)

    a = T1+1
    b = T2-1
    k = k*(i+1)

T1 = mu
T2 = mu+1
x, y = sujoy(img, a, T1)
w, z = sujoy(img, T2, b)    
T.append(x)
T.append(w)
T.sort()
print(T)

For full paper and more informations visit this link.