Detecting crosses in an image

Question

I am working on a program to detect the tips of a probing device and analyze the color change during probing. The input/output mechanisms are more or less in place. What I need

Answer 1

Template Matching Approach

Here is a simple matchTemplate solution, that is similar to the approach that Guy Sirton mentions.

Template matching will work as long as you don't have much scaling or rotation occurring with your target.

Here is the template that I used:

Here is the code I used to detect several of the unobstructed crosses:

#include 
#include 
#include 
#include 

using namespace cv;
using namespace std;

int main(int argc, char* argv[])
{
    string inputName = "crosses.jpg";
    string outputName = "crosses_detect.png";
    Mat img   = imread( inputName, 1);
    Mat templ = imread( "crosses-template.jpg", 1);

    int resultCols =  img.cols - templ.cols + 1;
    int resultRows = img.rows - templ.rows + 1;
    Mat result( resultCols, resultRows, CV_32FC1 );

    matchTemplate(img, templ, result, CV_TM_CCOEFF);
    normalize(result, result, 0, 255.0, NORM_MINMAX, CV_8UC1, Mat());

    Mat resultMask;
    threshold(result, resultMask, 180.0, 255.0, THRESH_BINARY);

    Mat temp = resultMask.clone();
    vector< vector > contours;
    findContours(temp, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, Point(templ.cols / 2, templ.rows / 2));

    vector< vector >::iterator i;
    for(i = contours.begin(); i != contours.end(); i++)
    {
        Moments m = moments(*i, false);
        Point2f centroid(m.m10 / m.m00, m.m01 / m.m00);
        circle(img, centroid, 3, Scalar(0, 255, 0), 3);
    }

    imshow("img", img);
    imshow("results", result);
    imshow("resultMask", resultMask);

    imwrite(outputName, img);

    waitKey(0);

    return 0;
}

This results in this detection image:

This code basically sets a threshold to separate the cross peaks from the rest of the image, and then detects all of those contours. Finally, it computes the centroid of each contour to detect the center of the cross.

Shape Detection Alternative

Here is an alternative approach using triangle detection. It doesn't seems as accurate as the matchTemplate approach, but might be an alternative you could play with.

Using findContours we detect all the triangles in the image, which results in the following:

Then I noticed all the triangle vertices cluster near the cross center, so then these clusters are used to centroid the cross center point shown below:

Finally, here is the code that I used to do this:

#include 
#include 
#include 
#include 
#include 

using namespace cv;
using namespace std;

vector getAllTriangleVertices(Mat& img, const vector< vector >& contours);
double euclideanDist(Point a, Point b);

vector< vector > groupPointsWithinRadius(vector& points, double radius);
void printPointVector(const vector& points);
Point computeClusterAverage(const vector& cluster);

int main(int argc, char* argv[])
{
    Mat img   = imread("crosses.jpg", 1);
    double resizeFactor = 0.5;
    resize(img, img, Size(0, 0), resizeFactor, resizeFactor);

    Mat momentImg = img.clone();

    Mat gray;
    cvtColor(img, gray, CV_BGR2GRAY);

    adaptiveThreshold(gray, gray, 255.0, ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 19, 15);
    imshow("threshold", gray);
    waitKey();

    vector< vector > contours;
    findContours(gray, contours, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);

    vector allTriangleVertices = getAllTriangleVertices(img, contours);

    imshow("img", img);
    imwrite("shape_detect.jpg", img);
    waitKey();

    printPointVector(allTriangleVertices);
    vector< vector > clusters = groupPointsWithinRadius(allTriangleVertices, 10.0*resizeFactor);
    cout << "Number of clusters: " << clusters.size() << endl;

    vector< vector >::iterator cluster;
    for(cluster = clusters.begin(); cluster != clusters.end(); ++cluster)
    {
        printPointVector(*cluster);

        Point clusterAvg = computeClusterAverage(*cluster);
        circle(momentImg, clusterAvg, 3, Scalar(0, 255, 0), CV_FILLED);
    }

    imshow("momentImg", momentImg);
    imwrite("centroids.jpg", momentImg);
    waitKey();

    return 0;
}

vector getAllTriangleVertices(Mat& img, const vector< vector >& contours)
{
    vector approxTriangle;
    vector allTriangleVertices;
    for(size_t i = 0; i < contours.size(); i++)
    {
        approxPolyDP(contours[i], approxTriangle, arcLength(Mat(contours[i]), true)*0.05, true);
        if(approxTriangle.size() == 3)
        {
            copy(approxTriangle.begin(), approxTriangle.end(), back_inserter(allTriangleVertices));
            drawContours(img, contours, i, Scalar(0, 255, 0), CV_FILLED);

            vector::iterator vertex;
            for(vertex = approxTriangle.begin(); vertex != approxTriangle.end(); ++vertex)
            {
                circle(img, *vertex, 3, Scalar(0, 0, 255), 1);
            }
        }
    }

    return allTriangleVertices;
}

double euclideanDist(Point a, Point b)
{
    Point c = a - b;
    return cv::sqrt(c.x*c.x + c.y*c.y);
}

vector< vector > groupPointsWithinRadius(vector& points, double radius)
{
    vector< vector > clusters;
    vector::iterator i;
    for(i = points.begin(); i != points.end();)
    {
        vector subCluster;
        subCluster.push_back(*i);

        vector::iterator j;
        for(j = points.begin(); j != points.end(); )
        {
            if(j != i &&  euclideanDist(*i, *j) < radius)
            {
                subCluster.push_back(*j);
                j = points.erase(j);
            }
            else
            {
                ++j;
            }
        }

        if(subCluster.size() > 1)
        {
            clusters.push_back(subCluster);
        }

        i = points.erase(i);
    }

    return clusters;
}

Point computeClusterAverage(const vector& cluster)
{
    Point2d sum;
    vector::const_iterator point;
    for(point = cluster.begin(); point != cluster.end(); ++point)
    {
        sum.x += point->x;
        sum.y += point->y;
    }

    sum.x /= (double)cluster.size();
    sum.y /= (double)cluster.size();

    return Point(cvRound(sum.x), cvRound(sum.y));
}

void printPointVector(const vector& points)
{
    vector::const_iterator point;
    for(point = points.begin(); point != points.end(); ++point)
    {
        cout << "(" << point->x << ", " << point->y << ")";
        if(point + 1 != points.end())
        {
            cout << ", ";
        }
    }
    cout << endl;
}

I fixed a few bugs in my previous implementation, and cleaned the code up a bit. I also tested it with various resize factors, and it seemed to perform quite well. However, after I reached a quarter scale it started to have trouble properly detecting triangles, so this might not work well for extremely small crosses. Also, it appears there is a bug in the moments function as for some valid clusters it was returning (-NaN, -NaN) locations. So, I believe the accuracy is a good bit improved. It may need a few more tweaks, but overall I think it should be a good starting point for you.

I think my triangle detection would work better if the black border around the triangles was a bit thicker/sharper, and if there were less shadows on the triangles themselves.

Hope that helps!