What causes precision to drop at IoU 0.5 for MaskRCNN?

Question

I am experimenting with different optimizers and learning rate schedulers to improve the performance of a MaskRCNN model (40K images input, instance segmentation, 1 class).