Concatenate two huge files in C++

Question

I have two std::ofstream text files of a hundred plus megs each and I want to concatenate them. Using fstreams to store the data to create a single file usually ends up with an

Answer 1

Assuming you don't want to do any processing, and just want to concatenate two files to make a third, you can do this very simply by streaming the files' buffers:

std::ifstream if_a("a.txt", std::ios_base::binary);
std::ifstream if_b("b.txt", std::ios_base::binary);
std::ofstream of_c("c.txt", std::ios_base::binary);

of_c << if_a.rdbuf() << if_b.rdbuf();

I have tried this sort of thing with files of up to 100Mb in the past and had no problems. You effectively let C++ and the libraries handle any buffering that's required. It also means that you don't need to worry about file positions if your files get really big.

An alternative is if you just wanted to copy b.txt onto the end of a.txt, in which case you would need to open a.txt with the append flag, and seek to the end:

std::ofstream of_a("a.txt", std::ios_base::binary | std::ios_base::app);
std::ifstream if_b("b.txt", std::ios_base::binary);

of_a.seekp(0, std::ios_base::end);
of_a << if_b.rdbuf();

---

How these methods work is by passing the std::streambuf of the input streams to the operator<< of the output stream, one of the overrides of which takes a streambuf parameter (operator<<). As mentioned in that link, in the case where there are no errors, the streambuf is inserted unformatted into the output stream until the end of file.