Deflate Compression Stream where pre compressed Data can be inserted. Does a .NET lib exist?

Question

I'm implementing Deflate and GZip compression for web content. The .NET Framework DeflateStream performs very well (it doesn't compress that good as SharpZipLib, but it is much faster). Unfortunately it (and all other libs i know) miss a function to write precompressed data like stream.WritePrecompressed(byte[] buffer).

With this function it would be possible to insert precompressed blocks in the stream. This could reduce the cpu load for compressing this part and increase the total throughput of the web server.

Is there any managed lib capable of doing this? Or is there any good starting point beyond ZLIB.NET from ComponentAce to do this?

Answer 1

Another approach is to flush the deflater stream (and possibly also close it), to guarantee that all buffered compressed data is written to the output stream, and then simply write your precompressed data to the underlying output stream, then re-open the deflater stream on top of your output stream again.

Answer 2

IIRC the #ZipLib allows you to set the compression level, have you tried flushing the stream and dropping the level to 0 and then sending the already compressed data before raising the compression level again?

If you are only looking at doing this for performance reasons then this might be an acceptable solution.

Answer 3

Yes, you can insert precompressed blocks in to a zlib stream. Start with the zpipe.c example in the zlib source. Only where you want to insert your precompressed block, replace Z_NO_FLUSH with Z_FULL_FLUSH (otherwise don't use Z_FULL_FLUSH because the compression ratio will suffer.)

Now the compressed output is byte aligned and the last deflate block is closed. Full flush means that the next block past the precompressed block cannot contain any back references.

Append your precompressed block to the output stream (e.g. memcpy). Advance strm.next_out to the next empty byte. Continue with deflate where you left off.

flush = feof(source) ? Z_FINISH : Z_NO_FLUSH;
ret = deflate(&strm, flush);