Recently I’ve been working on training from scratch (with no pre-training whatsoever) a RoBERTa model starting from the code in this tutorial.
I am working with a specifi
