Compilers: Understanding assembly code generated from small programs

Question

I\'m self-studying how compilers works. I\'m learning by reading the disassembly of GCC generated code from small 64-bit Linux programs.

I wrote this C

Answer 1

1. Yes, the nop is for alignment. Compilers use different instructions for different lengths of padding needed, knowing that modern CPU will be pre-fetching and decoding several instructions ahead.

2. As others have said, the C99 standard returns 0 from main() by default if there's no explicit return statement (see 5.1.2.2.3 in C99 TC3), so no warning is raised.

3. The 64-bit System V Linux ABI reserves a 128-byte "red zone" below the current stack pointer that leaf functions (functions that do not call any other functions - and your main() is one such) can use for local variables and other scratch values without having to sub rsp / add rsp. And so rbp == rsp.

And for the PS: when you call a function in the for() loop (or anywhere in your main()), main() is no longer a leaf function, so the compiler can no longer use the red zone. That's why the it allocates space on the stack with sub rsp, 0x10. However, it knows the relationship between rsp and rbp, so it can use either when accessing data.

Answer 2

Anything after the ret cannot be relied on to be code. Decoding as nop means "No OPeration"

The 2nd point is the compiler detecting you leave the main function without returning a value and it inserts a return 0 (only defined for main).

The rbp register, with bp meaning "Base Pointer", points to the stack frame of the currect function. A function call often results in the function entry saving rbp and using the current value of rsp for rbp. Fetching/storing function arguments and local variables are done relative to rbp.

---

I think your third question needs some more attention, "Why doesn't the compiler allocate space on the stack with sub rsp,0x10? Why doesn't it use the rbp register for referencing local stack data?"

Actually, the compiler does allocate space on the stack. But it does not change the stackpointer. It can do that because the functon calls no other functions. It just uses space below the curent sp (the stack grows down) and it uses rbp to access i ([rbp-0x8]) and k ([rbp-0x4]).

---

I must add the following note: not adjusting sp for the use of local variables seems not interrupt safe and so the compiler relies on the hardware automatically switching to a system stack when interrupts occur. Otherwise, the first interrupt that came along would push the instruction pointer onto the stack and would overwrite the local variable.

Question of interrupts solved in Compiler using local variables without adjusting RSP

Answer 3

Regarding the second question, since the C99 standard it's allowed to not have an explicit return 0 in the main function, the compiler will add it implicitly. Note that this is only for the main function, no other function.

As for the third question, the rbp register acts as the frame pointer.

Lastly the PS. It's likely that the called function is using 16 bytes (0x10) for the arguments passed to the function. The subtraction is what "removes" those variables from the stack. Could it possibly be two pointers you pass as arguments?

If you're serious learning how compilers in general works, and possibly want to create your own (it's fun! :)), then I suggest you invest in some books about the theory and practice of it. The dragon book is an excellent addition to any programmers bookshelf.