Switch from LBFGS to ADAM optimizer in the middle of training in pytorch

Question

I want to switch the optimizer in the middle of the training from LBFGS to Adam. Below is the code that I wrote.

optimizer = optim.LBFGS(model.parameters(), lr=0.