Zero a large memory mapping with `madvise`

Question

I have the following problem:

I allocate a large chunk of memory (multiple GiB) via mmap with MAP_ANONYMOUS. That chunk holds a large hash map w

Answer 1

There is a much easier solution to your problem that is fairly portable:

mmap(ptr, length, PROT_READ|PROT_WRITE, MAP_FIXED|MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

Since MAP_FIXED is permitted to fail for fairly arbitrary implementation-specific reasons, falling back to memset if it returns MAP_FAILED would be advisable.

Answer 2

This madvise behavior is certainly not standard, so this wouldn't be portable.

If the part that you want to zero out happens to be at the end of your mapping you could get away with ftruncate. You'd have to introduce one step more:

1. shm_open to have a "persistent" file descriptor to your data
2. ftruncate to the needed size
3. mmap of that FD

Then you could always

1. munmap
2. ftruncate to something short
3. ftruncate to the real length you need
4. mmap again

and then the part that you "remapped" would be zero initialized.

But have also in mind that the system has to do the zeroing of the pages. This might be a bit more efficient than the inline stuff that your compiler produces for memset, but that is not sure.

Answer 3

On Linux, you can rely on MADV_DONTNEED on an anonymous mapping zeroing the mapping. This isn't portable, though - madvise() itself isn't standardised. posix_madvise() is standardised, but the POSIX_MADV_DONTNEED does not have the same behaviour as the Linux MADV_DONTNEED flag - posix_madvise() is always advisory, and does not affect the semantics of the application.