I am trying to setup c# code to manage our Google domain.
I am receiving this error whenever I call service.Users.List() or any other method from the DirectoryServic
Scopes
It appears that you are trying this Quickstart:
However, the scope(s) used in that tuturoial are:
new [] { DirectoryService.Scope.AdminDirectoryUserReadonly };
However, in the code your posted code you have:
new[] { DirectoryService.Scope.AdminDirectoryOrgunit, DirectoryService.Scope.AdminDirectoryUser },
Tokens
After you change your scopes (shown above), you may have to delete your OAuth2 token, and then re-authorize access for your application. (Unless you haven't done the "authorize access" step yet.)
\token.json\Google.Apis.Auth.OAuth2.Responses.TokenResponse-user
Enable APIs
Also, as I think you already discovered, enabling the Directory API is different process than enabling the Gmail API (and found at different URLs)
Enable Directory API
Enable Gmail API