My answer uses an intermediate buffer, but perhaps you will be happy that it uses iterators to initialize the string from the buffer.
std::vector buffer(128); // create a buffer
ifs.read( &buffer[0], buffer.size() ); // read to buffer
std::string first128( buffer.begin(), buffer.end() ); // copy from vector
To me it seems like they got a little too cute with the implementation of iostreams. Trying to use iterators for stream I/O is overly complicated.
By the way, I suspect the implementation that you were attempting will, under the covers, do a variety of intermediate buffering (perhaps some in the kernel, some in the library) as well as re-allocating and copying the string several times as it grows.
One other idea: Do you really need the result in a standard string? You might just work from the vector -- avoiding the final step of copying to a string. Or, if you are feeling adventurous, you could create your own string class that does allow you to expose the internal buffer in the same way that vector does.