The main difference as I understand it is that in Debug Mode the whole symbol information which is used by the debugger is stored along with the program, so that if a developer wants to debug the application before releasing he/she may do so, by attaching to any debugger.
You might've noticed the .pdb files in the debug folder. Also the size of the executable is fairly larger. However in Release mode, the debugger symbol information is omitted assuming that the end user is going to use the application so he must not be provided with the application symbols.
You can think of the symbols as the information provided to the debugger to understand what local variables, what functions, where breakpoints are set and all sorts of information so that it can precisely tell you what part of code is being executed currently.
