I have a GSuite service account configured to access corporate user gmail accounts, I have provided it with all of the privileges in the G Suite Admin console including Domain W
Ok the answer was that you MUST specify a user when creating the credential. In the newer version of the API this call has been changed to: .createDelegated(). Just put the users email that you wish to impersonate there.
GoogleCredential credential = GoogleCredential.fromStream(resourceAsStream).createDelegated("deusrex@mygenericdomain.com");