Best Practice to Use HttpClient in Multithreaded Environment

Question

For a while, I have been using HttpClient in a multithreaded environment. For every thread, when it initiates a connection, it will create a completely new HttpClient instan

Answer 1

Definitely Method A because its pooled and thread safe.

If you are using httpclient 4.x, the connection manager is called ThreadSafeClientConnManager. See this link for further details (scroll down to "Pooling connection manager"). For example:

    HttpParams params = new BasicHttpParams();
    SchemeRegistry registry = new SchemeRegistry();
    registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
    ClientConnectionManager cm = new ThreadSafeClientConnManager(params, registry);
    HttpClient client = new DefaultHttpClient(cm, params);

Answer 2

My reading of the docs is that HttpConnection itself is not treated as thread safe, and hence MultiThreadedHttpConnectionManager provides a reusable pool of HttpConnections, you have a single MultiThreadedHttpConnectionManager shared by all threads and initialised exactly once. So you need a couple of small refinements to option A.

MultiThreadedHttpConnectionManager connman = new MultiThreadedHttpConnectionManag

Then each thread should be using the sequence for every request, getting a conection from the pool and putting it back on completion of its work - using a finally block may be good. You should also code for the possibility that the pool has no available connections and process the timeout exception.

HttpConnection connection = null
try {
    connection = connman.getConnectionWithTimeout(
                        HostConfiguration hostConfiguration, long timeout) 
    // work
} catch (/*etc*/) {/*etc*/} finally{
    if ( connection != null )
        connman.releaseConnection(connection);
}

As you are using a pool of connections you won't actually be closing the connections and so this should not hit the TIME_WAIT problem. This approach does assuume that each thread doesn't hang on to the connection for long. Note that conman itself is left open.

Answer 3

I think you will want to use ThreadSafeClientConnManager.

You can see how it works here: http://foo.jasonhudgins.com/2009/08/http-connection-reuse-in-android.html

Or in the AndroidHttpClient which uses it internally.

Answer 4

Method A is recommended by httpclient developer community.

Please refer http://www.mail-archive.com/httpclient-users@hc.apache.org/msg02455.html for more details.

Answer 5

With HttpClient 4.5 you can do this:

CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(new PoolingHttpClientConnectionManager()).build();

Note that this one implements Closeable (for shutting down of the connection manager).