How can I pre-train Transformers model like BERT, RoBERTa… from scratch on CPU instead of GPU or TPU?

Question

In terms of code and libraries which required for train such models, is it differ on CPU from GPU or TPU? I mean will I need to create some specific libraries? or all librar