Decompress file from Boost filtering_streambuf to std::vector<char>?

Question

I'm trying to decompress a file that was compressed using the DEFLATE algorithm and stuff it into a vector<unsigned char>. From the research I've done so far, it seemed like I could use a boost::iostreams::filtering_streambuf and then use boost::iostreams::copy() to get it into a boost::interprocess::basic_vectorstream<std::vector<unsigned char>> and then pull the underlying vector out of the vectorstream. However, I'm getting a ton of compiler errors, starting with this:

/usr/include/boost/iostreams/copy.hpp: In function ‘std::streamsize boost::iostreams::detail::copy_impl(Source, Sink, std::streamsize) [with Source = boost::reference_wrapper<boost::iostreams::filtering_streambuf<boost::iostreams::input> >, Sink = boost::reference_wrapper<boost::interprocess::basic_vectorstream<std::vector<unsigned char> > >, std::streamsize = long int]’:
/usr/include/boost/iostreams/copy.hpp:245:79:   instantiated from ‘std::streamsize boost::iostreams::copy(Source&, Sink&, std::streamsize, typename boost::enable_if<boost::iostreams::is_std_io<Source> >::type*, typename boost::enable_if<boost::iostreams::is_std_io<Sink> >::type*) [with Source = boost::iostreams::filtering_streambuf<boost::iostreams::input>, Sink = boost::interprocess::basic_vectorstream<std::vector<unsigned char> >, std::streamsize = long int, typename boost::enable_if<boost::iostreams::is_std_io<Source> >::type = void, typename boost::enable_if<boost::iostreams::is_std_io<Sink> >::type = void]’
ValueFileReader.cpp:92:41:   instantiated from here
/usr/include/boost/iostreams/copy.hpp:178:5: error: static assertion failed: "(is_same<src_char, snk_char>::value)"

The code I'm using is below (shortened up and doing "using namespace" for brevity here):

using namespace boost::iostreams;
using namespace boost::interprocess;

filtering_streambuf<input> in;
std::ifstream file(filename, std::ifstream::in);
in.push(zlib_decompressor());
in.push(file);

basic_vectorstream<std::vector<unsigned char>> vectorStream;
copy(in, vectorStream);
std::vector<unsigned char> chars(vectorStream.vector());

I've seen this thread, but I wasn't sure if copying everything into a vector, then decompressing from the vector would be the most efficient way of going about it.

Is there a better way I can go about this, or do I have the right idea but some error in the code?

Answer 1

The problem is that your ifstream and filtering_streambuf use char as their underlying character type, but your basic_vectorstream uses unsigned char as its value type. The Boost code has a static assertion requiring that these types be the same so that you don't get a spew of compiler errors if you use two different types that are not convertible.

Fortunately, the fix here is easy – change:

basic_vectorstream<std::vector<unsigned char>> vectorStream;
copy(in, vectorStream);
std::vector<unsigned char> chars(vectorStream.vector());

to:

basic_vectorstream<std::vector<char>> vectorStream;
copy(in, vectorStream);
std::vector<unsigned char> chars(
    vectorStream.vector().begin(),
    vectorStream.vector().end()
);

This is safe because char and unsigned char are guaranteed by the C++ standard to have the same object representation (§3.9.1/1).

---

Unrelated to your direct problem, but you also need to pass std::ios::binary to file's constructor, otherwise you'll have corrupt data due to line-ending conversions.