RuntimeError: cublas runtime error : the GPU program failed to execute at /pytorch/aten/src/THC/THCBlas.cu:450

Question

My program works properly with two GPUs but raise this error when reached to torch.bmm

energy = torch.bmm(proj_query,proj_key) # transpose check RuntimeError: cublas