I tried all of the above but didn't work for me. In the end noticed in my error message my call back was slightly different. I had a users between localhost:3000 and auth. Not really sure why.
http://localhost:3000/users/auth/google_oauth2/callback
Changed it, waited 30 mins and it worked.