Inreproducible loss/results when training bert with same setting for >=2 times

Question

I am using my customized bert script to train a model. However, everything even I keep the same setting for lr, AdamW weight decay and epoch, and run on the same platform (c