OpenCV - GrabCut with custom foreground/background models

Question

I want to use the GrabCut algorithm implemented on OpenCV.

As shown in the documentation this is the function signature:

void grabCut(
    InputArra         


        

Answer 1

In opencv/sources/modules/imgproc/src/grabcut.cpp you can have a look how the models (GMMs) are encoded:

GMM::GMM( Mat& _model )
{
    const int modelSize = 3/*mean*/ + 9/*covariance*/ + 1/*component weight*/;
    if( _model.empty() )
    {
        _model.create( 1, modelSize*componentsCount, CV_64FC1 );
        _model.setTo(Scalar(0));
    }
    else if( (_model.type() != CV_64FC1) || (_model.rows != 1) || (_model.cols != modelSize*componentsCount) )
        CV_Error( CV_StsBadArg, "_model must have CV_64FC1 type, rows == 1 and cols == 13*componentsCount" );

    model = _model;

    coefs = model.ptr<double>(0);
    mean = coefs + componentsCount;
    cov = mean + 3*componentsCount;

    for( int ci = 0; ci < componentsCount; ci++ )
        if( coefs[ci] > 0 )
             calcInverseCovAndDeterm( ci );
}

So you need for every model a cv::Mat of 1 x 65 doubles (componentsCount equals 5). There are 3 means per component because its computing in RGB colorspace. Using GC_EVAL would indeed leave the models intact but I never tried it with pre-computed models.

Answer 2

Even I had a similar problem. This is how I solved it. I edited the GC_EVAL condition in the grabcut source code to this -

if( mode == GC_EVAL )
   { checkMask( img, mask );
    for( int i = 0; i < iterCount; i++ )
{
    GCGraph<double> graph;
    assignGMMsComponents( img, mask, bgdGMM, fgdGMM, compIdxs );

    constructGCGraph(img, mask, bgdGMM, fgdGMM, lambda, leftW, upleftW, upW, uprightW, graph );
    estimateSegmentation( graph, mask );
    return;    
}
}

Notice that the function learnGMMs is not called here. This is done because the Foreground and Background GMMs are precomputed.

You can save models in a .xml file using the following code snippet.

 FileStorage fs("mymodels.xml", FileStorage::WRITE);
        fs << "BgdModel" << bgdModel;
        fs << "FgdModel" << fgdModel;
fs.release();

You can retrieve the models using the following code.

FileStorage fs1("mymodels.xml", FileStorage::READ);

        fs1["BgdModel"] >> bgdModel1;

        fs1["FgdModel"] >> fgdModel1;

This worked for me.