Office.js does not always load on dialog page reload in my Excel-Addin

Question

I was trying to implement a basic auth flow on the lines of https://docs.microsoft.com/en-us/office/dev/add-ins/develop/auth-with-office-dialog-api. After completing my auth