Avoid Circular Redirect using HttpClient 4.1.1

Question

How can I avoid circular redirect using HttpClient 4.1.1. As I am getting the error like this:-

executing requestGET http://home.somehost.com/Mynet/pages/cHo         


        

Answer 1

There is a bug that will cause circular redirect in Apache HttpClient since 4.0, it wasn't fixed even in the latest version.

In DefaultRequestDirector.java, it creates a HttpRedirect to perform redirection, and it will reuse all headers in your original HttpGet, the problem here is it will also reuse Host header, which mean the server will still get the original host after it's attempt to redirect to new URI.

I fixed this by reimplemented the DefaultRequestDirector:

public class RedirectRequestDirector extends DefaultRequestDirector
{
    RedirectRequestDirector(
            final HttpRequestExecutor requestExec,
            final ClientConnectionManager conman,
            final ConnectionReuseStrategy reustrat,
            final ConnectionKeepAliveStrategy kastrat,
            final HttpRoutePlanner rouplan,
            final HttpProcessor httpProcessor,
            final HttpRequestRetryHandler retryHandler,
            final RedirectHandler redirectHandler,
            final AuthenticationHandler targetAuthHandler,
            final AuthenticationHandler proxyAuthHandler,
            final UserTokenHandler userTokenHandler,
            final HttpParams params) 
    {
        super(requestExec, conman, reustrat, kastrat, rouplan, httpProcessor, retryHandler, redirectHandler, targetAuthHandler, proxyAuthHandler, userTokenHandler, params);

    }
    @Override
    protected RoutedRequest handleResponse(RoutedRequest roureq,
            HttpResponse response,
            HttpContext context)
                    throws HttpException, IOException
    {
        RoutedRequest req = super.handleResponse(roureq, response, context);
        if(req != null)
        {
            String redirectTarget = req.getRoute().getTargetHost().getHostName();
            req.getRequest().getOriginal().setHeader("Host", redirectTarget);
        }
        return req;
    }

}

and DefaultHttpClient:

public class RedirectHttpClient extends DefaultHttpClient
{
    @Override
    protected RequestDirector createClientRequestDirector(
            final HttpRequestExecutor requestExec,
            final ClientConnectionManager conman,
            final ConnectionReuseStrategy reustrat,
            final ConnectionKeepAliveStrategy kastrat,
            final HttpRoutePlanner rouplan,
            final HttpProcessor httpProcessor,
            final HttpRequestRetryHandler retryHandler,
            final RedirectHandler redirectHandler,
            final AuthenticationHandler targetAuthHandler,
            final AuthenticationHandler proxyAuthHandler,
            final UserTokenHandler stateHandler,
            final HttpParams params) {
        return new RedirectRequestDirector(
                requestExec,
                conman,
                reustrat,
                kastrat,
                rouplan,
                httpProcessor,
                retryHandler,
                redirectHandler,
                targetAuthHandler,
                proxyAuthHandler,
                stateHandler,
                params);
    }
}

Now I won't complain about the Circular Redirect.

Answer 2

Check that your request isnt sent to a proxy before being sent to the url you requested.

Answer 3

I faced this issue while spring version upgrade, the context is not initialized properly in my case.

In org.apache.http.impl.client.DefaultRedirectStrategy:

RedirectLocations redirectLocations = (RedirectLocations) clientContext.getAttribute(
                HttpClientContext.REDIRECT_LOCATIONS);

The value of the clientContext should be basicHttpContext, but Spring Web (4.3.x.RELEASE) is initializing the context in:

org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal();

The value of clientContext is changing, which results in circular redirect error. The Spring Web (3.2.x.RELEASE) don't initialize the context and value will be null.

Answer 4

You just avoided it. HttpClient detected the circular redirect and threw an exception. Had it not been "avoided", it would continue redirecting forever (until you decided to kill the process). There aren't a whole lot of other options, if that's what the server responds with.

The only way to truly avoid a circular redirect loop is to fix the server.

If you are wondering what is going on (like why it seems to work find in a browser but not from your program), try turning on some of the extra HttpClient logging. In particular, make sure you can see all of the HTTP headers being sent back and forth. You can then look at the conversation taking place when you make the same request in your browser, noting the differences. It could be a missing cookie, crazy browser detection, etc...

There are a number of ways of tracing your browser's communications. Here are a few ways that I often use, in order from easiest to hardest (IMHO):

• Firefox + HttpFox (or LiveHttpHeaders, Firebug, etc...)
• Fiddler (Windows only)
• Wireshark/tcpdump

For low-level testing, try using telnet (unless you use Windows, in which case you may be better off with something like PuTTY/plink) and ruling in/out what changes cause the circular redirects.

Answer 5

You can set the ClientPNames.ALLOW_CIRCULAR_REDIRECTS to true, this will allow redirects to the same location.

  client.getParams().setParameter(ClientPNames.ALLOW_CIRCULAR_REDIRECTS, true); 

See more info here

Answer 6

You may try:

RequestConfig requestConfig = RequestConfig.custom()
                              .setCircularRedirectsAllowed(true)
                              .build();

HttpClient httpClient = HttpClients.custom()
                        .setDefaultRequestConfig(requestConfig)
                        .setRedirectStrategy(new LaxRedirectStrategy())
                        .build();

HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);