Why would my REST service .NET clients send every request without authentication headers and then retry it with authentication header?

Question

We happen to run a REST web service with API requiring that clients use Basic authentication. We crafted a set of neat samples in various languages showing how to inter

Answer 1

This is the default behavior of HttpClient and HttpWebRequest classes which is exposed the following way.

Note: Below text explains suboptimal behavior causing the problem described in the question. Most likely you should not write your code like this. Instead scroll below to the corrected code

In both cases, instantiate a NetworkCredenatial object and set the username and password in there

var credentials = new NetworkCredential( username, password );

If you use HttpWebRequest - set .Credentials property:

webRequest.Credentials = credentials;

If you use HttpClient - pass the credentials object into HttpClientHandler (altered code from here):

var client = new HttpClient(new HttpClientHandler() { Credentials = credentials })

Then run Fiddler and start the request. You will see the following:

• the request is sent without Authorization header
• the service replies with HTTP 401 and WWW-Authenticate: Basic realm="UrRealmHere"
• the request is resent with proper Authorization header (and succeeds)

This behavior is explained here - the client doesn't know in advance that the service requires Basic and tries to negotiate the authentication protocol (and if the service requires Digest sending Basic headers in open is useless and can compromise the client).

Note: Here suboptimal behavior explanation ends and better approach is explained. Most likely you should use code from below instead of code from above.

For cases when it's known that the service requires Basic that extra request can be eliminated the following way:

Don't set .Credentials, instead add the headers manually using code from here. Encode the username and password:

var encoded = Convert.ToBase64String( Encoding.ASCII.GetBytes(
    String.Format( "{0}:{1}", username, password ) ) );

When using HttpWebRequest add it to the headers:

request.Headers.Add( "Authorization", "Basic " + encoded );

and when using HttpClient add it to default headers:

client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue( "Basic", encoded );

When you do that the request is sent with the right authorization headers every time. Note that you should not set .Credentials, otherwise if the username or password is wrong the same request will be sent twice both time with the wrong credentials and both times of course yielding HTTP 401.