Pytorch: revert weight trained using apex back to normal float32

Question

Background: I downloaded a weight that is trained using pytorch_pretrained_bert==0.4.0 plus apex, and now I want to load it to a framework that has no