400 error with HttpClient for a link with an anchor

Question

Here is my code:

DefaultHttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
HttpResponse response = client.execute(request);
         


        

Answer 1

As @Greg Sansom says, the URL should not be sent with an anchor / fragment. The fragment part of the URL is not relevant to the server.

Here's the expected URL syntax from relevant part of the HTTP 1.1 specification:

    http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]] 

Note: there is no fragment part in the syntax.

What happens if you do send a fragment clearly is server implementation specific. I expect that you will see a variety of responses:

• Some servers will silently strip / ignore the fragment part. (This is what you are expecting to happen).
• Some servers might treat this as a request error and respond with a 400.
• Some servers might mistakenly treat the fragment as part of the path or query, and give you a 404 or some other response, depending on how "confused" the fragment makes the server.
• Some servers might actually imbue the fragment with a specific meaning. (This strikes me as a stupid thing to do, but you never know ...)

IMO, the most sensible solution is to strip it from the URL before instantiating the HttpGet object.

FOLLOWUP

The recommended way to remove a fragment from a URL string is to turn it into a java.net.URL or java.net.URI instance, extract the relevant components, use these to create a new java.net.URL or java.net.URI instance (leaving out the fragment of course), and finally turn it back into a String.

But I think that the following should also work, if you can safely assume that your URLs are all valid absolute HTTP or HTTPS URLs.

    int pos = url.indexOf("#");
    String strippedUrl = (pos >= 0) ? url.substring(0, pos) : url;