How to give a threshold to the transformation matrix to avoid dramatic transformation?

Question

I want to align points coordinates, with 3 sets of reference and input points, then calculate the transformation matrix obtained by skimage.transform.SimilarityTransform.