To much fluctuation while estimating pose of camera, using Apriltag and inverse of 3 x 3 Rotation pose matrix

Question

First of all, I am explaining, what I am doing,

I am using apriltag c++ python wrapper to estimate the pose of the TAG. The outputs are Pose 3 x 3 Rotation matrix an