How to zip- compress HTTP request with Spring RestTemplate?

Question

How to gzip HTTP request, created by org.springframework.web.client.RestTemplate?

I am using Spring 4.2.6 with Spring

Answer 1

Since I cannot comment on @roj 's post I'm writing an answer here.

@roj snippet although is neat it actually does not do the same job as @Testo Testini 's snippet.

Testo is closing the streams before: byteStream.toByteArray(); where in @rog answer, this occurs before the stream.close(), since streams are in the try/resource block.

If you need to use try-with-resources, zipStream should be closed before the byteStream.toByteArray();

The complete snippet should be:

private byte[] getGzip(byte[] body) throws IOException {

    try (ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
         GZIPOutputStream zipStream = new GZIPOutputStream(byteStream)) {
            
        zipStream.write(body);
        zipStream.close();

        byte[] compressedData = byteStream.toByteArray();
        return compressedData;
    }

}

The was getting an error ("Compressed file ended before the end-of-stream marker was reached") and the above fixed the error in my case and I thought that I should share this.

Answer 2

I propose two solutions, one simpler without streaming and one that supports streaming.

If you don't require streaming, use a custom ClientHttpRequestInterceptor, a Spring feature.

RestTemplate rt = new RestTemplate();
rt.setInterceptors(Collections.singletonList(interceptor));

Where interceptor could be:

ClientHttpRequestInterceptor interceptor = new ClientHttpRequestInterceptor() {

    @Override
    public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution)
            throws IOException {
        request.getHeaders().add("Content-Encoding", "gzip");
        byte[] gzipped = getGzip(body);
        return execution.execute(request, gzipped);
    } 
 }

getGzip I copied

    private byte[] getGzip(byte[] body) throws IOException {

        ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
        try {
            GZIPOutputStream zipStream = new GZIPOutputStream(byteStream);
            try {
                zipStream.write(body);
            } finally {
                zipStream.close();
            }
        } finally {
            byteStream.close();
        }

        byte[] compressedData = byteStream.toByteArray();
        return compressedData;

    }

After configuring the interceptor all requests will be zipped.

The disadvantage of this approach is that it does not support streaming as the ClientHttpRequestInterceptor receives the content as a byte[]

If you require streaming create a custom ClientHttpRequestFactory, say GZipClientHttpRequestFactory, and use it like this:

    SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
    requestFactory.setBufferRequestBody(false);
    ClientHttpRequestFactory gzipRequestFactory = new GZipClientHttpRequestFactory(requestFactory);
    RestTemplate rt = new RestTemplate(gzipRequestFactory);

Where GZipClientHttpRequestFactory is:

public class GZipClientHttpRequestFactory extends AbstractClientHttpRequestFactoryWrapper {

    public GZipClientHttpRequestFactory(ClientHttpRequestFactory requestFactory) {
        super(requestFactory);
    }

    @Override
    protected ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod, ClientHttpRequestFactory requestFactory)
            throws IOException {
        ClientHttpRequest delegate = requestFactory.createRequest(uri, httpMethod);
        return new ZippedClientHttpRequest(delegate);
    }

}

And ZippedClientHttpRequest is:

public class ZippedClientHttpRequest extends WrapperClientHttpRequest
{
    private GZIPOutputStream zip;

    public ZippedClientHttpRequest(ClientHttpRequest delegate) {
        super(delegate);
        delegate.getHeaders().add("Content-Encoding", "gzip");
        // here or in getBody could add content-length to avoid chunking
        // but is it available ? 
        // delegate.getHeaders().add("Content-Length", "39");

    }

    @Override
    public OutputStream getBody() throws IOException {
        final OutputStream body = super.getBody();
        zip = new GZIPOutputStream(body);
        return zip;
    }

    @Override
    public ClientHttpResponse execute() throws IOException {
        if (zip!=null) zip.close();
        return super.execute();
    }

}

And finally WrapperClientHttpRequest is:

public class WrapperClientHttpRequest implements ClientHttpRequest {

    private final ClientHttpRequest delegate;

    protected WrapperClientHttpRequest(ClientHttpRequest delegate) {
        super();
        if (delegate==null)
            throw new IllegalArgumentException("null delegate");
        this.delegate = delegate;
    }

    protected final ClientHttpRequest getDelegate() {
        return delegate;
    }

    @Override
    public OutputStream getBody() throws IOException {
        return delegate.getBody();
    }

    @Override
    public HttpHeaders getHeaders() {
        return delegate.getHeaders();
    }

    @Override
    public URI getURI() {
        return delegate.getURI();
    }

    @Override
    public HttpMethod getMethod() {
        return delegate.getMethod();
    }

    @Override
    public ClientHttpResponse execute() throws IOException {
        return delegate.execute();
    }
}

This approach creates a request with chunked transfer encoding, this can be changed setting the content length header, if size is known.

The advantage of the ClientHttpRequestInterceptor and/or custom ClientHttpRequestFactory approach is that it works with any method of RestTemplate. An alternate approach, passing a RequestCallback is possible only with execute methods, this because the other methods of RestTemplate internally create their own RequestCallback(s) that produce the content.

BTW it seems that there is little support to decompress gzip request on the server. Also related: Sending gzipped data in WebRequest? that points to the Zip Bomb issue. I think you will have to write some code for it.

Answer 3

Further to the above answer from @TestoTestini, if we take advantage of Java 7+'s 'try-with-resources' syntax (since both ByteArrayOutputStream and GZIPOutputStream implement closeable() ) then we can shrink the getGzip function into the following:

private byte[] getGzip(byte[] body) throws IOException {

    try (ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
         GZIPOutputStream zipStream = new GZIPOutputStream(byteStream)) {
            zipStream.write(body);
        byte[] compressedData = byteStream.toByteArray();
        return compressedData;
    }

}

(I couldn't find a way of commenting on @TestoTestini's original answer and retaining the above code format, hence this Answer).