Does Java HTTP Client handle compression

Question

I tried to find any mention of handling of compression in new Java HTTP Client but failed. Is there a built-in configuration to handle for e.g. gzip or de

Answer 1

No, gzip/deflate compression are not handled by default. You would have to implement that in your application code if you need it - e.g. by providing a customized BodySubscriber to handle it. Alternatively - you may want to have a look at whether some of the reactive stream libraries out there offer such a feature, in which case you might be able to pipe that in by using one of the BodyHandlers.fromSubscriber​(Flow.Subscriber<? super List<ByteBuffer>> subscriber) or BodyHandlers.ofPublisher() methods.

Answer 2

This question is a bit old, but I recently released a library that fixes this issue. The library is called Methanol and it's available on Maven.

You can use it to decode the response as follows:

HttpResponse<String> response = client.send(request, MoreBodyHandlers.decoding(BodyHandlers.ofString()));

You can also use any BodyHandler you want. MoreBodyHandlers::decoding makes it seem to your handler like the response was never compressed! It takes care of the Content-Encoding header and all. Gzip and deflate are supported by default, there's also a module for brotli.

Better yet, you can use the custom HttpClient for transparent compression (not needing to add Accept-Encoding):

Methanol client = Methanol.newBuilder()
    .autoAcceptEncoding(true) // note that true is the default
    .build();
HttpRequest request = ...
HttpResponse<String> response = client.send(request, BodyHandlers.ofString()); // response is compressed transparently

Answer 3

I was also surprised that the new java.net.http framework doesn't handle this automatically, but the following works for me to handle HTTP responses which are received as an InputStream and are either uncompressed or compressed with gzip:

public static InputStream getDecodedInputStream(
        HttpResponse<InputStream> httpResponse) {
    String encoding = determineContentEncoding(httpResponse);
    try {
        switch (encoding) {
            case "":
                return httpResponse.body();
            case "gzip":
                return new GZIPInputStream(httpResponse.body());
            default:
                throw new UnsupportedOperationException(
                        "Unexpected Content-Encoding: " + encoding);
        }
    } catch (IOException ioe) {
        throw new UncheckedIOException(ioe);
    }
}

public static String determineContentEncoding(
        HttpResponse<?> httpResponse) {
    return httpResponse.headers().firstValue("Content-Encoding").orElse("");
}

Note that I've not added support for the "deflate" type (because I don't currently need it, and the more I read about "deflate" the more of a mess it sounded). But I believe you can easily support "deflate" by adding a check to the above switch block and wrapping the httpResponse.body() in an InflaterInputStream.