mmap and memory usage

Question

I am writing a program that receives huge amounts of data (in pieces of different sizes) from the network, processes them and writes them to memory. Since some pieces of data ca

Answer 1

I believe mmap doesn't require all data to be in memory at the same moment - it uses the page cache to keep recently used pages in memory, and the rest on disk.

If you are reading one chunk at a time, using a temporary file probably won't help you, but if you are reading multiple chunks concurrently using multiple threads, processes, or using select/poll, then it might.