OpenCV RotatedRect with specified angle

Question

I have the situation that I have a small binary image that has one shape, around which I want to find the best fitting rotated rectangle (not b

Answer 1

OK, my solution: Approach:

1. PCA, gives the angle and a first approximation for the rotatedRect's center
2. Get the contour of the binary shape, rotate it into upright position, get min/max of X and Y coordinates to get the width and height of the bounding rect
3. Subtract half the width (height) from maximum X (Y) to get the center point in the "upright space"

4. Rotate this center point back by the inverse rotation matrix

   cv::RotatedRect Utilities::getBoundingRectPCA( cv::Mat& binaryImg ) {
   cv::RotatedRect result;
   
   //1. convert to matrix that contains point coordinates as column vectors
   int count = cv::countNonZero(binaryImg);
   if (count == 0) {
       std::cout << "Utilities::getBoundingRectPCA() encountered 0 pixels in binary image!" << std::endl;
       return cv::RotatedRect();
   }
   
   cv::Mat data(2, count, CV_32FC1);
   int dataColumnIndex = 0;
   for (int row = 0; row < binaryImg.rows; row++) {
       for (int col = 0; col < binaryImg.cols; col++) {
           if (binaryImg.at<unsigned char>(row, col) != 0) {
               data.at<float>(0, dataColumnIndex) = (float) col; //x coordinate
               data.at<float>(1, dataColumnIndex) = (float) (binaryImg.rows - row); //y coordinate, such that y axis goes up
               ++dataColumnIndex;
           }
       }
   }
   
   //2. perform PCA
   const int maxComponents = 1;
   cv::PCA pca(data, cv::Mat() /*mean*/, CV_PCA_DATA_AS_COL, maxComponents);
   //result is contained in pca.eigenvectors (as row vectors)
   //std::cout << pca.eigenvectors << std::endl;
   
   //3. get angle of principal axis
   float dx = pca.eigenvectors.at<float>(0, 0);
   float dy = pca.eigenvectors.at<float>(0, 1);
   float angle = atan2f(dy, dx)  / (float)CV_PI*180.0f;
   
   //find the bounding rectangle with the given angle, by rotating the contour around the mean so that it is up-right
   //easily finding the bounding box then
   cv::Point2f center(pca.mean.at<float>(0,0), binaryImg.rows - pca.mean.at<float>(1,0));
   cv::Mat rotationMatrix = cv::getRotationMatrix2D(center, -angle, 1);
   cv::Mat rotationMatrixInverse = cv::getRotationMatrix2D(center, angle, 1);
   
   std::vector<std::vector<cv::Point> > contours;
   cv::findContours(binaryImg, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
   if (contours.size() != 1) {
       std::cout << "Warning: found " << contours.size() << " contours in binaryImg (expected one)" << std::endl;
       return result;
   }
   
   //turn vector of points into matrix (with points as column vectors, with a 3rd row full of 1's, i.e. points are converted to extended coords)
   cv::Mat contourMat(3, contours[0].size(), CV_64FC1);
   double* row0 = contourMat.ptr<double>(0);
   double* row1 = contourMat.ptr<double>(1);
   double* row2 = contourMat.ptr<double>(2);
   for (int i = 0; i < (int) contours[0].size(); i++) {
       row0[i] = (double) (contours[0])[i].x;
       row1[i] = (double) (contours[0])[i].y;
       row2[i] = 1;
   }
   
   cv::Mat uprightContour = rotationMatrix*contourMat;
   
   //get min/max in order to determine width and height
   double minX, minY, maxX, maxY;
   cv::minMaxLoc(cv::Mat(uprightContour, cv::Rect(0, 0, contours[0].size(), 1)), &minX, &maxX); //get minimum/maximum of first row
   cv::minMaxLoc(cv::Mat(uprightContour, cv::Rect(0, 1, contours[0].size(), 1)), &minY, &maxY); //get minimum/maximum of second row
   
   int minXi = cvFloor(minX);
   int minYi = cvFloor(minY);
   int maxXi = cvCeil(maxX);
   int maxYi = cvCeil(maxY);
   
   //fill result
   result.angle = angle;
   result.size.width = (float) (maxXi - minXi);
   result.size.height = (float) (maxYi - minYi);
   
   //Find the correct center:
   cv::Mat correctCenterUpright(3, 1, CV_64FC1);
   correctCenterUpright.at<double>(0, 0) = maxX - result.size.width/2;
   correctCenterUpright.at<double>(1,0) = maxY - result.size.height/2;
   correctCenterUpright.at<double>(2,0) = 1;
   cv::Mat correctCenterMat = rotationMatrixInverse*correctCenterUpright;
   cv::Point correctCenter = cv::Point(cvRound(correctCenterMat.at<double>(0,0)), cvRound(correctCenterMat.at<double>(1,0)));
   
   result.center = correctCenter;
   
   return result;
   

   }

Answer 2

If understand the problem correctly, you're saying the method of using findContours and minAreaRectsuffers from jitter/wobbling due to the noisy input data. PCA is not more robust against this noise, so I don't see why you think finding the orientation of the pattern this way won't be as bad as your current code.

If you need temporal smoothness a commonly used and simple solution is to use a filter, even a very simple filter like an alpha-beta filter probably gives you the smoothness you want. Say at frame n you estimate the parameters of the rotated rectangle A, and in frame n+1 you have the rectangle with the estimated parameters B. Instead of drawing the rectangle with B you find C which is between A and B, and then draw a rectangle with C in frame n+1.

Answer 3

Here's another approach (just a guess)

Wikipedia page on Principal Component Analysis says:

PCA can be thought of as fitting an n-dimensional ellipsoid to the data ...

And as your data is 2D, you can use the cv::fitEllipse function to fit an ellipse to your data and use the coordinates of the generated RotatedRect to calculate the angle. This gives better results as compared to cv::minAreaRect.