Why is MAP_GROWSDOWN mapping does not grow?

Question

I tried to create MAP_GROWSDOWN mapping with the expectation it would grow automatically. As specified in the manual page:

MAP_GROWSDOWN

This flag is used for stacks. It indicates to the kernel virtual memory system that the mapping should extend downward in memory. The return address is one page lower than the memory area that is actually created in the process's virtual address space. Touching an address in the "guard" page below the mapping will cause the mapping to grow by a page. This growth can be repeated until the mapping grows to within a page of the high end of the next lower mapping, at which point touching the "guard" page will result in a SIGSEGV signal.

So I wrote the following example to test the mapping growing:

#ifndef _GNU_SOURCE
    #define _GNU_SOURCE
#endif
#include <stdlib.h>
#include <string.h>
#include <inttypes.h>
#include <errno.h>
#include <sys/mman.h>
#include <stdio.h>

int main(void){
    char *mapped_ptr = mmap(NULL, 4096,
                            PROT_READ | PROT_WRITE,
                            MAP_ANONYMOUS | MAP_PRIVATE | MAP_STACK | MAP_GROWSDOWN,
                            -1, 0);
    if(mapped_ptr == MAP_FAILED){
        int error_code = errno;
        fprintf(stderr, "Cannot do MAP_FIXED mapping."
                        "Error code = %d, details = %s\n", error_code, strerror(error_code));
                        exit(EXIT_FAILURE);
    }
    volatile char *c_ptr_1 = mapped_ptr; //address returned by mmap
    *c_ptr_1 = 'a'; //fine

    volatile char *c_ptr_2 = mapped_ptr - 4095; //1 page below the guard
    *c_ptr_2 = 'b'; //crashes with SEGV
}

So I got SEGV instead of growing the mapping. What does it mean by growing here?

Answer 1

Replace:

volatile char *c_ptr_1 = mapped_ptr - 4096; //1 page below

With

volatile char *c_ptr_1 = mapped_ptr;

Because:

The return address is one page lower than the memory area that is actually created in the process's virtual address space. Touching an address in the "guard" page below the mapping will cause the mapping to grow by a page.

Note that I tested the solution and it works as expected on kernel 4.15.0-45-generic.

Answer 2

First of all, you don't want MAP_GROWSDOWN, and it's not how the main thread stack works. Analyzing memory mapping of a process with pmap. [stack] Nothing uses it, and pretty much nothing should use it. The stuff in the man page saying it's "used for stacks" is wrong and should be fixed.

I suspect it might be buggy (because nothing uses it so usually nobody cares or even notices if it breaks.)

---

Your code works for me if I change the mmap call to map more than 1 page. Specifically, I tried 4096 * 100. I'm running Linux 5.0.1 (Arch Linux) on bare metal (Skylake).

/proc/PID/smaps does show a gd flag.

And then (when single-stepping the asm) the maps entry does actually change to a lower start address but the same end address, so it is literally growing downward when I start with a 400k mapping. This gives a 400k initial allocation above the return address, which grows to 404kiB when the program runs. (The size for a _GROWSDOWN mapping is not the growth limit or anything like that.)

https://bugs.centos.org/view.php?id=4767 may be related; something changed between kernel versions in CentOS 5.3 and 5.5. And/or it had something to do with working in a VM (5.3) vs. not growing and faulting on bare metal (5.5).

---

I simplified the C to use ptr[-4095] etc:

int main(void){
    volatile char *ptr = mmap(NULL, 4096*100,
                            PROT_READ | PROT_WRITE,
                            MAP_ANONYMOUS | MAP_PRIVATE | MAP_STACK | MAP_GROWSDOWN,
                            -1, 0);
    if(ptr == MAP_FAILED){
        int error_code = errno;
        fprintf(stderr, "Cannot do MAP_FIXED mapping."
                        "Error code = %d, details = %s\n", error_code, strerror(error_code));
                        exit(EXIT_FAILURE);
    }

    ptr[0] = 'a';      //address returned by mmap
    ptr[-4095] = 'b';  // grow by 1 page
}

Compiling with gcc -Og gives asm that's nice-ish to single-step.

---

BTW, various rumours about the flag having been removed from glibc are obviously wrong. This source does compile, and it's clear that it's also supported by the kernel, not silently ignored. (Although the behaviour I see with size 4096 instead of 400kiB is exactly consistent with the flag being silently ignored. However the gd VmFlag is still there in smaps, so it's not ignored at that stage.)

I checked and there was room for it to grow without coming close to another mapping. So IDK why it didn't grow when the GD mapping was only 1 page. I tried a couple times and it segfaulted each time. With the larger initial mapping it never faulted.

Both times were with a store to the mmap return value (the first page of the mapping proper), then a store 4095 bytes below that.