opencv-solvepnp

问题 We are working on an AR application in which we need to overlay a 3D model of an object on a video stream of the object. A Unity scene contains the 3D model and a camera is filming the 3D object. The camera pose is initially unknown. ▶ What we have tried We did not find a good solution to estimate the camera pose directly in Unity. We, therefore, used OpenCV which provides an extensive library of computer vision functions. In particular, we locate Aruco tags and then pass their matching 3D-2D

问题 Found solution at https://github.com/opencv/opencv/issues/4943 OpenCV Error: Assertion failed (CV_IS_MAT(_src) && CV_IS_MAT(_dst) && (_src->rows == 1 || _src->cols == 1) && (_dst->rows == 1 || _dst->cols == 1) && _src->cols + _src->rows - 1 == _dst->rows + _dst->cols - 1 && (CV_MAT_TYPE(_src->type) == CV_32FC2 || CV_MAT_TYPE(_src->type) == CV_64FC2) && (CV_MAT_TYPE(_dst->type) == CV_32FC2 || CV_MAT_TYPE(_dst->type) == CV_64FC2)) in cvUndistortPoints, file /home/javvaji/opencv-3.2.0/modules

问题 Could somebody help me with the math on calculating the Y rotation of my camera? I use SolvePnP to get a rvec. From this rvec I want to know how much my camera is rotated around its own Y axis. PointF[] ImagePointsCam = GetImagePointsCam(imgThres); MCvPoint3D32f[] ObjectPointsCam = GetObjectPointsCam(); float f = 1.0f / (float)(Math.Tan(CameraFOV / 360.0 * 3.1415)); Matrix<float> CamMat = new Matrix<float>(3, 3); CamMat.Data[0, 0] = (float)(f * 0.5 * ResolutionWidth); CamMat.Data[0, 1] = 0;

问题 I get strange data from solvePnP, so I tried to check it with projectPoints: retval, rvec, tvec=cv2.solvePnP(opts, ipts, mtx, dist, flags=cv2.SOLVEPNP_ITERATIVE) print(retval,rvec,tvec) proj, jac = cv2.projectPoints(opts, rvec, tvec, mtx, dist) print(proj,ipts) here opts are 3d points with z=0, detected on this picture: And ipts are taken from this pic (only part of picture here): I've checked points themselves (detected with SIFT, points are detected correctly and pairing in a right way).