This applies to both 2008 and 2010 versions (and probably earlier ones too). Also to both C++ and C# projects.
An initial build (after reboot) runs at normal speed
I had the same problem.
I had McAfee security center installed, by disabling the "Real time scan"
Building times did go from 40 seconds for a small project to 1 second.