My situation
In the C# project I am now working on we have a fairly big solution (80+ projects). Now rebuild times of 5 minutes+ are really becoming qui
I'm working on 500+ C# application projects. Projects are compiled in parallel and copylocal set to false. Compile time is about 37 min without unit tests and code coverage. 13 min for incremental build without any change in the code.
If I turn off parallel compilation and set copylocal to true, compile time is than 1 h 40 min.
I have different configuration for local build, gated check-in build and server builds with deploy phase (night builds).
Here are my experiences:
/m:2
build should be 2x faster as a /m:1
build. It has nothing to do with dependencies between projects (if copylocal is set to false).Yes, MSBuild uses a timestamp of dependent projects to determine if a project needs a rebuild. It compares input files (code files, referenced assemblies, temporary files,..) timestamp with output assembly. If something is changed, your project is recompiled. Try to reduce the number of dependencies between projects to minimize recompilation. If your change was only in the 'private' part of the project, your output assembly will be changed, assembly timestamp will be changed and all related projects will be rebuild also. You cannot do much with this.
Run your build two times with diagnostic verbosity without any change in your code and check for "Building target "CoreCompile" completely" like I described here. You can have something wrong in your project files and your projects are recompiled every time. If you don't change anything your build log should not contain "Building target "CoreCompile" completely" logs.
Our build server is a virtual machine, not a real piece of hardware. It is not good idea to use a VM for a build server, but it was not my decision.
If you have multi GB RAM try to use part of it as a in-memory hard drive. Your build should be much faster :)
SSD drives are sensitive to high I/O per day. It has an impact on warranty.
I hope it helps someone ... ;)
If you got enough RAM and are using Visual C++ (not sure about C#), you could accelerate things by copying the whole include and lib directory to a RAM drive.
You can also place the temporary build items to be on the RAM drive. Of course you need a massive amount of RAM. A 2 GB RAM drive would be enough for the includes and libraries. But for the temporary files (*.obj, etc.) it would depend on the project. So it might be between 1 and 4 GB extra or more.
We also have huge solutions. Build and compilation is all about I/O.
Solid-state drives are very promising. A co-worker put a solid-state drive in his laptop, and found that it is now much faster than his humongous main development box. Don't have the details, but he claims many times faster.
We've been fiddling with solution folders to group parts of the project: This makes it easier for devs to unload projects they aren't working on.
/m
rarely helps with .NET debug builds. I ran a series of tests on this a few weeks ago and found minor differences. Findings:
Big picture: In reality, compilation is a pretty minor cost for me, compared to getting source and running unit tests. On our build server, the integration tests and packaging are almost all the time. For my development box, I schedule a batch file that gets source, builds, runs unit tests before I come to work and while I'm at lunch. Good enough.
On the build server, it's more complicated. We're thinking of setting up chained parallel CruiseControl.NET builds on various machines. We are using VSTS build, but it is too expensive (and time consuming) to scale horizontally like this.
My numbers for detail-oriented folks. Configurations from slowest to fastest, running msbuild "bigsolution.sln" /target:clean - between each.
Build time without rebuild or clean: ~ 4-10 seconds
MSBuild build time is a multi-dimensional problem. The good news that it is easy to solve:
Unlike most of the processes running on the build machine, build processes are notorious for being CPU, RAM and I/O -consuming. A general recipe for speeding up MSBuild builds is "get best machine money can buy", particularly:
CPU - at least two Intel 3.0 GHz Core 2 Duo.
RAM - at least 4 GB DDR3. If this is a developer and build machine, a 64-bit OS and 8 GB RAM is a better option.
HDD - The fastest options is a high-end 3ware RAID-1 with an on-board battery and an enabled write cache. A fast SSD may be another option to consider.
Network - minimum 1 Gbit/s card.
This simple hardware optimization can speed up your MSBuilds 2-3 times.