Mosaicing face problems in opencv

问题

Hi I am using opencv and c++. I have a left profile and a front face of the same individual. If after determining the transformation matrix between the face images and applying that transformation matrix to the left image,when I superimpose it on the original frontal face,shouldn’t it have given me a face like this

,with everything aligned?I am obviously doing something wrong and getting this result

. Can anyone help with this please?Here is the link to that research paper http://www.utdallas.edu/~herve/abdi-ypaa-jmm2006.pdf

回答1:

taken the images from one of your previous questions, manually finding pixel correspondences for the green dots only (so 3 correspondences) and this code

//cv::Mat perspectiveTransform = cv::getPerspectiveTransform(firstFacePositions, secondFacePositions) ;
cv::Mat affineTransform = cv::getAffineTransform(firstFacePositions, secondFacePositions) ;
std::cout << affineTransform << std::endl;
cv::Mat perspectiveTransform = cv::Mat::eye(3,3,CV_64FC1);
for(unsigned int y=0; y<2; ++y)
    for(unsigned int x=0; x<3; ++x)
    {
        perspectiveTransform.at<double>(y,x) = affineTransform.at<double>(y,x);
    }

std::cout << perspectiveTransform << std::endl;

cv::Mat warped1;
cv::warpPerspective(face1,warped1,perspectiveTransform,face2.size());

cv::imshow("combined",warped1/2 + face2/2);

I get the following result:

using the line cv::Mat perspectiveTransform = cv::getPerspectiveTransform(firstFacePositions, secondFacePositions); instead and using the blue marker too, I get:

edit: C++ syntax, but that works with C# and Java similar I guess.

回答2:

The registration procedure is maybe working, but you need to define better which features you want to use for the alignment or you are using images with a deformation that is too large.

In the small (color) image, the eyes are aligned.
In the big (grey) image, parts of the face appear to be correctly aligned, but not the eyes (for example).

If your goal is to have eyes and mouth aligned, then you should detect them and apply some 2D (affine or homographic) mapping that aligns them with your template. if your goal is to have the green / blue points aligned, then you need to use them to feed the 2D mapping estimator, but in this case there will be no guarantee that the eyes, mouth, etc. are aligned: only the points should match.

来源：https://stackoverflow.com/questions/21695290/mosaicing-face-problems-in-opencv

标签

c++

OpenCV

computer-vision

mosaic