So to understand new/delete better (really to prove to myself with small examples why virtual destructors are needed for interfaces), I want to understand memory leaks, so that
Delete doesn't invalidate the pointer. It releases the memory it points to back to the heap. In this case, the memory hasn't been reused and therefore still contains the same content. At some point that memory WILL be reused and then you get the undefined behavior that others speak of.
This isn't a memory leak though. That's when you point the pointer to another memory allocation (or just discard the pointer) without freeing the memory it points to. The first then stays allocated and, as you've no references to it, you've no way of freeing it.