Segfault on stack overflow

Question

Why does the linux kernel generate a segfault on stack overflow? This can make debugging very awkward when alloca in c or fortran creation of temporary arrays overflows. Sur

Answer 1

You can actually catch the condition for a stack overflow using signal handlers.

To do this, you must do two things:

• Setup a signal handler for SIGSEGV (the segfault) using sigaction, to do this set the SO_ONSTACK flag. This instructs the kernel to use an alternative stack when delivering the signal.

• Call sigaltstack() to setup the alternate stack that the handler for SIGSEGV will use.

Then when you overflow the stack, the kernel will switch to your alternate stack before delivering the signal. Once in your signal handler, you can examine the address that caused the fault and determine if it was a stack overflow, or a regular fault.