How to optimize reading and writing by playing with buffer size?

Question

How to optimize reading and writing speed on std::ifstream and std::ofstream in standard C++/C++11 (no POSIX functions) ? (1 <- as there is several questions, these numbe

Answer 1

Your description of how the buffering works is correct, AFAICS.

A buffer size larger than 1 MB is unlikely to buy you anything, though. In reality, the sweet spot is likely far below that value. Note that the buffers used by std::ifstream and std::ofstream have nothing to do with the disk cache - that's the kernel's job, and it does it at its own discretion. The stream buffer only affect the maximum number of bytes transferred to or from the kernel with one syscall. Thus, the ideal buffer size doesn't depend on how much data you transfer. What is does depend on is

1. The overhead cost of syscalls. Higher overhead means you'll want to transfer more data in one go.
2. The overhead cost of buffer management. Probably larger for larger buffers, if anything.
3. CPU cache trashing effects. Will strongly favour smaller buffers.

Since (1) works in favour of larger buffers while (2) and (3) work in favour of smaller buffers, there will be a sweet spot somewhere. Since the CPU cache size is likely to be a couple of megabytes or so, a buffer size which approaches that limit will see a severe effect from (3), so the sweet spot surely lies beneath 1 MB or so. You can probably ignore (2), so it remains to estimate (1) to get a lower bound on the buffer size. Assume that a syscall costs somewhere around 1000 cycles or so, and assume that raw copying speed of CPU+memory is 4 bytes/cycle. Transferring 4k then costs about as much as doing one syscall. So with a buffer size of 20k, syscall overhead is roughly 20%, and at a buffer size of 100k it's about 4%. he ideal buffer size is thus somewhere in the range of a couple of hundred kB independent of the file size!.

You can probably trust your standard library implementation to get that right, unless profiling provides you with hard evidence that there is a buffering issue which affects performance.