Compiler using local variables without adjusting RSP

Question

In question Compilers: Understanding assembly code generated from small programs the compiler uses two local variables without adjusting the stack pointer.

Not adjus

Answer 1

This is the so called "red zone" of the x86-64 ABI. A summary from wikipedia:

In computing, a red zone is a fixed-size area in a function's stack frame beyond the current stack pointer which is not preserved by that function. The callee function may use the red zone for storing local variables without the extra overhead of modifying the stack pointer. This region of memory is not to be modified by interrupt/exception/signal handlers. The x86-64 ABI used by System V mandates a 128-byte red zone which begins directly under the current value of the stack pointer.

In 64-bit Linux user code it is OK, as long as no more than 128 bytes are used. It is an optimization used most prominently by leaf-functions, i.e. functions which don't call other functions,

---

If you were to compile the example program as a 64-bit Linux program with GCC (or compatible compiler) using the -mno-red-zone option you'd see code like this generated:

main:
        push    rbp
        mov     rbp, rsp
        sub     rsp, 16;     <<============  Observe RSP is now being adjusted.
        mov     DWORD PTR [rbp-4], 0
.L3:
        cmp     DWORD PTR [rbp-4], 9
        jg      .L2
        mov     DWORD PTR [rbp-8], 0
        add     DWORD PTR [rbp-4], 1
        jmp     .L3
.L2:
        mov     eax, 0
        leave
        ret

This code generation can be observed at this godbolt.org link.

---

For a 32-bit Linux user program it would be a bad thing not to adjust the stack pointer. If you were to compile the code in the question as 32-bit code (using -m32 option) main would appear something like the following code:

main:
        push    ebp
        mov     ebp, esp
        sub     esp, 16;     <<============  Observe ESP is being adjusted.
        mov     DWORD PTR [ebp-4], 0
.L3:
        cmp     DWORD PTR [ebp-4], 9
        jg      .L2
        mov     DWORD PTR [ebp-8], 0
        add     DWORD PTR [ebp-4], 1
        jmp     .L3
.L2:
        mov     eax, 0
        leave
        ret

This code generation can be observed at this gotbolt.org link.