Azure DocumentDB Intermittent 401 error when querying REST API via Obj-c

Question

I've been charged with implementing an objective-c based iOS query of the Azure DocumentDB system using the REST API scheme. Utilizing the code found on github, specifically https://github.com/Azure/azure-storage-ios I was able to generate a request that appropriately authenticates and returns the appropriate data.... sometimes.

The problem: I receive a 401 (authentication failure) error response from the server intermittently. Making the same request via Node.js does not encounter this behavior, so I believe this to be an issue with my objective-c implementation.

- (NSMutableURLRequest *) RequestWithQuery:(NSString*)query Parameters:(NSArray*)parameters {

NSError* error;
NSDictionary* dictionaryOfBodyContents = @{@"query":query,
                                           @"parameters":parameters};
NSData* body = [NSJSONSerialization dataWithJSONObject:dictionaryOfBodyContents
                                               options:NSJSONWritingPrettyPrinted
                                                 error:&error];

if(error != nil) {
    NSLog(@"AzureRequestWithQueryParameters error generating the body: %@",error);
    return nil;
}

char buffer[30];
struct tm * timeptr;

time_t time = (time_t) [[NSDate date] timeIntervalSince1970];
timeptr = gmtime(&time);
if (!strftime_l(buffer, 30, [@"%a, %d %b %Y %T GMT" UTF8String], timeptr, NULL))
{
    NSException* myException = [NSException
                                exceptionWithName:@"Error in date/time format"
                                reason:@"Unknown"
                                userInfo:nil];
    @throw myException;
}
NSString* date = [NSString stringWithUTF8String:buffer];
// generate auth token
NSString* authorizationToken = [self AuthorizationTokenForTableQueryWithDate:date];

// generate header contents
NSDictionary* dictionaryOfHeaderContents = @{@"authorization":authorizationToken,
                                             @"connection":AZURE_CONNECTION_HEADER_CONNECTION,
                                             @"content-type":AZURE_CONNECTION_HEADER_CONTENTTYPE,
                                             @"content-length":[NSString stringWithFormat:@"%lu",(unsigned long)[body length]],
                                             @"x-ms-version":AZURE_CONNECTION_APIVERSION,
                                             @"x-ms-documentdb-isquery":@"true",
                                             @"x-ms-date":date.lowercaseString,
                                             @"cache-control":@"no-cache",
                                             @"user-agent":AZURE_CONNECTION_HEADER_USERAGENT,
                                             @"accept":@"application/json"};

// generate url contents
NSString* urlString = [NSString stringWithFormat:@"https://%@:%@/%@", AZURE_URL_HOST, AZURE_URL_PORT, AZURE_URL_DOCUMENTS];
NSURL* url = [NSURL URLWithString:urlString];

NSMutableURLRequest* request = [[NSMutableURLRequest alloc] initWithURL:url];
[request setHTTPMethod:AZURE_CONNECTION_METHOD];
[request setAllHTTPHeaderFields:dictionaryOfHeaderContents];
[request setHTTPBody:body];
return request;
}

- (NSString*) AuthorizationTokenForTableQueryWithDate:(NSString*)date {
//
//  Based on https://msdn.microsoft.com/en-us/library/azure/dd179428.aspx under "Table Service (Shared Key Authentication)"
//
//    generating a authentication token is a Hash-based Message Authentication Code (HMAC) constructed from the request
//      and computed by using the SHA256 algorithm, and then encoded by using Base64 encoding.
//
//    StringToSign =  VERB + "\n" +
//                    Content-MD5 + "\n" +
//                    Content-Type + "\n" +
//                    Date + "\n" +
//                    CanonicalizedHeaders +
//                    CanonicalizedResource;
//
NSString* StringToSign = [NSString stringWithFormat:@"%@\n%@\n%@\n%@\n\n",
                          AZURE_CONNECTION_METHOD.lowercaseString?:@"",
                          AZURE_RESOURCE_TYPE.lowercaseString?:@"",
                          AZURE_URL_COLLECTIONS.lowercaseString?:@"",
                          date.lowercaseString?:@""];

// Generate Key/Message pair
NSData* keyData = [[NSData alloc] initWithBase64EncodedString:AZURE_AUTH_KEY options:NSDataBase64DecodingIgnoreUnknownCharacters];
NSData* messageData = [StringToSign dataUsingEncoding:NSUTF8StringEncoding];

// Encrypt your Key/Message using HMAC SHA256
NSMutableData* HMACData = [NSMutableData dataWithLength:CC_SHA256_DIGEST_LENGTH];
CCHmac(kCCHmacAlgSHA256, keyData.bytes, keyData.length, messageData.bytes, messageData.length, HMACData.mutableBytes);

// Take your encrypted data, and generate a token that Azure likes.
NSString* signature = [HMACData base64EncodedStringWithOptions:NSDataBase64Encoding64CharacterLineLength];
NSString* unencodedToken = [NSString stringWithFormat:@"type=master&ver=1.0&sig=%@",signature];
NSString* authorizationToken = [unencodedToken stringByReplacingOccurrencesOfString:@"&" withString:@"%26"];
authorizationToken = [authorizationToken stringByReplacingOccurrencesOfString:@"=" withString:@"%3D"];

return authorizationToken;
}

If anyone has encountered a similar intermittent 401 and was able to resolve any help would be appreciated. Or suggestions for debugging steps for the above code bearing in mind, I have attempted decrementing the timestamp by a few seconds, similar intermittent failures.

Although simply retrying a few times upon a failure while decrementing the seconds results in a 200 response in 1-2 retries, I don't feel it is an ideal solution by any means.

Thank you for your time.

Update: Please see Andrew Liu's explanation below for the reason for this failure. I have flagged his response as the answer, below is the updated snippet of code.

NSString* unencodedToken = [NSString stringWithFormat:@"type=master&ver=1.0&sig=%@",signature];
//    NSString* authorizationToken = [unencodedToken stringByReplacingOccurrencesOfString:@"&" withString:@"%26"];
//    authorizationToken = [authorizationToken stringByReplacingOccurrencesOfString:@"=" withString:@"%3D"];
NSString* authorizationToken = [unencodedToken stringByAddingPercentEncodingWithAllowedCharacters:[[NSCharacterSet characterSetWithCharactersInString:@"&+="] invertedSet]];
return authorizationToken;

Answer 1

401 (auth failure) usually indicates that something is wrong with the auth token.

It's important to note that the auth token is a Base64-encoded string - meaning it can contain the + character.

The db server expects + characters in the auth token to be url encoded (%2B)... some but not all HTTP clients will automatically encode HTTP headers for you.

I suspect url-encoding or converting + to %2B for the following variable will fix your intermittent 401 issue:

NSString* signature = [HMACData base64EncodedStringWithOptions:NSDataBase64Encoding64CharacterLineLength];

Answer 2

I've seen this issue before and usually it has to do with the final 2 steps of the protocol, i.e. either the base64 encoding is whacky, or the URI encoding. One way to debug this would be to print the auth token that you had sent, in case of failure, and see if there are any strange characters which are possibly not being transmitted correctly. You can post the buggy token here and I can take a look.