So to understand new/delete better (really to prove to myself with small examples why virtual destructors are needed for interfaces), I want to understand memory leaks, so that
Dereferencing a deleted pointer is undefined behavior, as already explained, i.e. you are doing something wrong. Compilers may help you find that error by changing the pointer's value to some magic value that will always cause dereferencing the pointer to fail and can give you a clue that it's because it was previously deleted.
A quick test showed the following behavior for me:
MSVC2010:
debug build: *P1 = 0xfeeefeee
release build: *P1 =
GCC 4.6:
debug & release builds: *P1 = 0
So you were just (un)lucky that you got again the original value. I recommend you to always build in debug mode for testing your programs (even if you aren't debugging) because it adds additional sanity checks to your code.