Tensorflow-gpu version 2.3, cuda toolkit version 10.1, cudnn version 7.6.5, Anaconda environment. The code used is the code from transformer tutorial on tensorflow official webs