Force retry on specific http status code

Question

Dealing with a specific website, sometimes I receive a http response with status code 403. I wanted to re-execute the request in such cases (because, in my specific situation, this server throws a 403 when it is actually overloaded). I tried to use a ResponseHandler together with a StandardHttpRequestRetryHandler, but it didn't work the way I hoped; I expected that throwing an exception in the ResponseHandler would trigger the StandardHttpRequestRetryHandler, but it does not seems to be the case. How can I achieve the desired functionality?

Here is a sample code that illustrates my situation:

import java.io.IOException;

import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.client.methods.RequestBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.StandardHttpRequestRetryHandler;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;


public class Main {

  public static void main(String[] args) throws Exception {

    // a response handler that throws an exception if status is not 200
    ResponseHandler<String> responseHandler = new ResponseHandler<String> () {

      @Override
      public String handleResponse(HttpResponse response) throws
        ClientProtocolException, IOException
      {
        System.out.println("-> Handling response");

        if (response.getStatusLine().getStatusCode() != 200){
          // I expected this to trigger the retryHandler 
          throw new ClientProtocolException("Status code not supported");
        }
        return EntityUtils.toString(response.getEntity());
      }

    };

    StandardHttpRequestRetryHandler retryHandler = 
        new StandardHttpRequestRetryHandler(5, true)
    {
      @Override
      public boolean retryRequest(
          final IOException exception,
          final int executionCount,
          final HttpContext context)
      {
        System.out.println("-> Retrying request");
        return super.retryRequest(exception, executionCount, context);
      }
    };

    // my client with my retry handler
    HttpClient client = HttpClients
        .custom()
        .setRetryHandler(retryHandler)
        .build();

    // my request
    HttpUriRequest request = RequestBuilder
        .create("GET")
        .setUri("http://httpstat.us/403")         //always returns 403
        .build();

    String contents = client.execute(request, responseHandler);

    System.out.println(contents);
  }


}

Answer 1

Try using a custom ServiceUnavailableRetryStrategy

CloseableHttpClient client = HttpClients.custom()
        .setServiceUnavailableRetryStrategy(new ServiceUnavailableRetryStrategy() {
            @Override
            public boolean retryRequest(
                    final HttpResponse response, final int executionCount, final HttpContext context) {
                int statusCode = response.getStatusLine().getStatusCode();
                return statusCode == 403 && executionCount < 5;
            }

            @Override
            public long getRetryInterval() {
                return 0;
            }
        })
        .build();

Answer 2

You can do it by manually checking the status code, something like this:

CloseableHttpResponse response = null;
boolean success = false;
while(!success) {
    response = client.execute(httpGet);
    int status = response.getStatusLine().getStatusCode();
    success = (status == 200);
    if (!success) {
        if(status == 403) {
            Thread.sleep(2000); // wait 2 seconds before retrying
        } else {
            throw new RuntimeException("Something went wrong: HTTP status: " + status);
        }
    }
}
String contents = EntityUtils.toString(response.getEntity());
response.close();

// ....

System.out.println(contents);

You would need to add some things like retrying a predefined number of times before throwing a final exception and catching some checked exceptions (like the InterruptedException thrown by Thread.sleep()), but basically the code shows the main idea.