Setting time out in apache http client

Question

I'm using Apache http client 4.3.2 to send get requests. What I have done is:

private final RequestConfig requestConfig = RequestConfig.custom()
        .setConnectTimeout(1000)
        .setConnectionRequestTimeout(1000)
        .setSocketTimeout(1000)
        .build();
private final HttpClient client = HttpClientBuilder.create()
        .disableAuthCaching()
        .disableAutomaticRetries()
        .disableConnectionState()
        .disableContentCompression()
        .disableCookieManagement()
        .disableRedirectHandling()
        .setDefaultRequestConfig(requestConfig)
        .build(); 

And when sending request:

HttpGet request = null;

try {
    request = new HttpGet(url);
    if (client.execute(request).getStatusLine().getStatusCode() == 200) {
        /* do some work here */
    }
} catch (Exception e) {
    Logger.error(e);
} finally {
    if (request != null) {
        request.releaseConnection();
    }
}

But some of my requests still takes long time to timeout. This is stack trace of exception:

java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:152)
    at java.net.SocketInputStream.read(SocketInputStream.java:122)
    at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
    at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
    at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
    at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
    at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
    at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
    at com.sun.proxy.$Proxy0.receiveResponseHeader(Unknown Source)
    at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)

Is there any other time out value I should set? What am I doing wrong?

Answer 1

When I had this problem I changed my request to configure the timeout on each request.

//...
HttpRequestBase request = new HttpGet(url); //or HttpPost

RequestConfig.Builder requestConfig = RequestConfig.custom();
requestConfig.setConnectTimeout(30 * 1000);
requestConfig.setConnectionRequestTimeout(30 * 1000);
requestConfig.setSocketTimeout(30 * 1000);

request.setConfig(requestConfig.build());

CloseableHttpResponse response = client.execute(request);
//...

It worked fine.

Answer 2

Here is a code snippet example to achieve the goal :

int timeout = 5;
RequestConfig config = RequestConfig.custom()
  .setConnectTimeout(timeout * 1000)
  .setConnectionRequestTimeout(timeout * 1000)
  .setSocketTimeout(timeout * 1000).build();
CloseableHttpClient client =  HttpClientBuilder.create().setDefaultRequestConfig(config).build();

That is the recommended way of configuring all three timeouts in a type-safe and readable manner.

Here is the full detail

Answer 3

You can try to abort the request with HttpUriRequest#abort(), see https://hc.apache.org/httpcomponents-client-4.3.x/httpclient/apidocs/org/apache/http/client/methods/HttpUriRequest.html#abort%28%29. However, setting a timemout that requires no intercepting would be nicer.

Answer 4

.setSocketTimeout(1000) denotes 1 second of inactivity in application in reading the data packets causes this exception.

You can test this by running the code in debug mode and delay moving to next step for some time.

Check what is the maximum possible time limit you would expect your application to take for data read operation, for example 50 seconds (extreme case for bulk data)--> then update .setSocketTimeout(50000)