Python 2.7/OpenCV 3.3: Error in cv2.initUndistortRectifyMap . Not showing undistort rectified images

Question

I want to distort and rectified my stereo images. For which I used Opencv 3.3 in Python 2.7. Code I used is :

import cv2
import numpy as np

cameraMatrixL = np.load('mtx_left.npy')
distCoeffsL = np.load('dist_left.npy')
cameraMatrixR = np.load('mtx_right.npy')
distCoeffsR = np.load('dist_right.npy')
R = np.load('R.npy')
T = np.load('T.npy')

imgleft = cv2.imread('D:\python\camera calibration and 3d const\left\left60.png',0)
imgright = cv2.imread('D:\python\camera calibration and 3d const\Right/right60.png',0)


R1,R2,P1,P2,Q,validPixROI1, validPixROI2 = cv2.stereoRectify(cameraMatrixL,distCoeffsL,cameraMatrixR,distCoeffsR,(640,480),R,T,alpha=1)

print Q
# distort images
undistort_map1, rectify_map1 = cv2.initUndistortRectifyMap(cameraMatrixL, distCoeffsR, R1, P1, (640,480), cv2.CV_16SC2)
undistort_map2, rectify_map2 = cv2.initUndistortRectifyMap(cameraMatrixR, distCoeffsR, R2, P2, (640,480), cv2.CV_16SC2)

undistor_output1 = cv2.remap(imgleft, undistort_map1, rectify_map1, cv2.INTER_LINEAR)
undistor_output2 = cv2.remap(imgright, undistort_map2, rectify_map2, cv2.INTER_LINEAR)

cv2.imshow('undistor_output1',undistor_output1)
cv2.imshow('undistor_output2',undistor_output2)
while (True):
 if cv2.waitKey(1) & 0xFF == ord('q'):
    break

I calibrated my camera separately and then used those obtained matrices into cv2.stereoRectify to get R1,R2,P1,P2,Q matrices which I used in cv2.initUndistortRectifyMap. But I am not getting undistort images correctly. It looks like this:

My Matrices are :

Q
[[  1.00000000e+00   0.00000000e+00   0.00000000e+00  -3.23097469e+02]
 [  0.00000000e+00   1.00000000e+00   0.00000000e+00  -2.40008609e+02]
 [  0.00000000e+00   0.00000000e+00   0.00000000e+00  -7.47885268e+00]
 [  0.00000000e+00   0.00000000e+00  -1.53249612e-02   0.00000000e+00]]



   cameraMartix Left
[[ 807.24668269    0.          326.78961645]
 [   0.          620.70299534  259.9187458 ]
 [   0.            0.            1.        ]]

camearMatrix Right
[[ 567.37537971    0.          278.76995505]
 [   0.          558.21417195  216.22972643]
 [   0.            0.            1.        ]]

Rotation
[[ 0.99973813 -0.02260904  0.00353613]
 [ 0.02269951  0.99934817 -0.02807079]
 [-0.00289917  0.0281437   0.99959968]]

Tranlation
[[-93.46968934]
 [ -1.48741179]
 [ 24.98692133]]

I have read many answers ut nothing solve my problem.

Straightforward solution on how to stereo calibration and rectifications OpenCV?

*undistortPoints in OpenCV (cv2) with Python wrong results

cv2.remap creates nothing

using initUndistortRectifyMap to undistort image points

opencv cv2.remap creates strange images*

What is the solution for this problem?

Thanks.

Answer 1

After trying out many things I found out a solution to this particular problem. After comparing camera rotation matrices and distortion matrices obtained using Python OpenCV and Matlab, I found out that there is much difference between matrices obtained using Python and Matlab. Therefore, I again calibrate my cameras and do stereo calibration using Python and OpenCV. Below figure shows the difference between old matrices (got from Python) and new matrices (got after calibrate again using Python).

In this figure, we can see that there is much difference between distortion coefficient matrix for right and left camera for old case (dist coeff. right old and left old) as compare to distortion coefficient matrix for right and left camera for new case (dist coeff. right new and left new). For right camera, old coefficients were 83.85 and -19.18 against 0.78 and -0.61 respectively. Similarly, for left camera, old coefficients were 123.7 and -1641.4 against -0.38 and 0.73 respectively. Also, first element (1,1) of old left camera matrix was 807.24 which differ much as compare to right camera old matrix. New left camera matrix has this element value as 552.41 which I think is correct. When I used all these new matrices and change my alpha value to 0 (alpha = 0) I got the following result.

For stereoCalbirate I used following inputs

retVal, cm1, dc1, cm2, dc2, r, t, e, f = cv2.stereoCalibrate(objpoints, imgpointsL, imgpointsR, cm1, dc1, cm2, dc2, (640, 480), None, None, cv2.CALIB_FIX_INTRINSIC, criteria)

where criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.001)