I had the following configuration:
var oAuthServerOptions = new OAuthAuthorizationServerOptions() { AllowInsecureHttp = true,