In addition to splattne's answer, I believe the reason for having it (and not cleaning it up after the build) is to support incremental compilation. If you've already compiled 100 classes and change one of them, it's much more efficient to just recompile the code for the one changed class and reassemble the exe/dll from a mixture of the new and old code.
Of course, incremental compilation is a lot more complex than just that - it has to keep track of everything so it can detect when it needs to recompile a class even if that class itself hasn't changed. (e.g. if a new overload becomes available in a class - some callers may need to be recompiled.)
