I use the BERT transformer on my QA model in PyTorch. Then I tried to add an LSTM layer on top of Bert.
The model I use is as follows:
class model(nn.Modul