Azure Search RetryPolicy

Question

We are using azure search and need to implement a retry stratgey as well as storing the Ids of failed documents as described.

Is there any documentation/samples on how to implement a RetryPolicy strategy in Azure Search.

Thanks

Answer 1

This is what I used:

 private async Task<DocumentIndexResult> IndexWithExponentialBackoffAsync(IndexBatch<IndexModel> indexBatch)
 {
      return await Policy
           .Handle<IndexBatchException>()
           .WaitAndRetryAsync(5, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)), (ex, span) =>
           {
                indexBatch = ((IndexBatchException)ex).FindFailedActionsToRetry(indexBatch, x => x.Id);
           })
           .ExecuteAsync(async () => await _searchClient.IndexAsync(indexBatch));
 }

It uses the Polly library to handle exponential backoff. In this case I use a model IndexModel that has a id field named Id. If you like to log or store the ids of the failed attempts you can do that in the WaitAndRetryAsync function like

((IndexBatchException)ex)ex.IndexingResults.Where(r => !r.Succeeded).Select(r => r.Key).<Do something here>

Answer 2

There is currently no sample showing how to properly retry on IndexBatchException. However, there is a method you can use to make it easier to implement: IndexBatchException.FindFailedActionsToRetry. This method extracts the IDs of failed documents from the IndexBatchException, correlates them with the actions in a given batch, and returns a new batch containing only the failed actions that need to be retried.

Regarding the rest of the retry logic, you might find this code in the ClientRuntime library useful. You will need to tweak the parameters based on the characteristics of your load. The important thing to remember is that you should use exponential backoff before retrying to help your service recover, since otherwise your requests may be throttled.