How to speed up g++ compile time (when using a lot of templates)

Question

This question is perhaps somehow odd, but how can I speed up g++ compile time? My C++ code heavily uses boost and templates. I already moved as much as possible out of the h

Answer 1

I assume that we are talking about minutes to compile a file, i.e. precompiled headers or local disk issues aren't the problem.

Long compilation times with deep template code (boost etc.) is often rooted in the unfriendly asymptotic behavior of gcc when it comes to template instantiation, in particular when variadic templates are emulated with template default arguments.

Here's a document which names reduced compilation time as a motivation for variadic templates:

• http://osl.iu.edu/~dgregor/cpp/variadic-templates.html

cpptruths had an article about how gcc-4.5 is much better in this behalf and how it does brilliantly with its variadic templates:

• http://cpptruths.blogspot.com/2010/03/faster-meta-programs-using-gcc-45-and.html

IIRC then BOOST has a way to limit the generation of template default parameters for the pseudo-variadics, I think 'g++ -DBOOST_MPL_LIMIT_LIST_SIZE=10' should work (the default is 20)

UPDATE: There is also a nice thread with general techniques to speed up compiling here on SO which might be useful:

• What techniques can be used to speed up C++ compilation times?

UPDATE: This one is about the performance issues when compiling templates, the accepted answer recommends gcc-4.5 too, also clang is mentioned as a positive example:

• Are there optimized c++ compilers for template use?

Answer 2

Try the PIMPL technique, this question: What techniques can be used to speed up C++ compilation times?

It'll prevent the compiler from following the chain of header files and implementations every time you need to do something.

Answer 3

Usually, the most expensive parts of compilation are (a) reading the source files (ALL of them) and (b) loading the compiler into memory for each source file.

If you have 52 source (.cc) files, each of which #includes 47 #include (.h) files, you are going to load the compiler 52 times, and you are going to plow through 2496 files. Depending on the density of comments in the files, you may be spending a fair chunk of time eating useless characters. (In one organization I have seen, header files varied between 66% and 90% comments, with only 10%-33% of the file being "meaningful". The single best thing that could be done to enhance readability of those files was strip out every last comment, leaving only code.)

Take a long look at how your program is physically organized. See whether you can combine source files, and simplify your hierarchy of #include files.

Decades ago, companies like IBM understood this, and would write their compilers so that the compiler could be handed a list of files to compile, not just one file, and the compiler would only be loaded once.

Answer 4

This paper describes a method for compiling template code much like "traditional" non-template object files. Saves compile & link time, with only one line of code overhead per template instantiation.

Answer 5

Instantiate less templates and inline functions. Precompile as much as you can and just link it rather than compiling everything from scratch. Make sure you're using the latest version of GCC.

However, it's a simple fact that C++ is an incredibly complex language and compiling it takes quite some time.

Answer 6

On top of what everybody else added and what you're already doing (parallelized build, compiler options, etc), consider hiding templates in implementation classes, accessed through interfaces. That means that instead of having a class like:

// ClsWithNoTemplates.h file, included everywhere

class ClsWithTemplates
{
    ComplicatedTemplate<abc> member;
    // ...

public:
    void FunctionUsingYourMember();
};

you should have:

// ClsWithNoTemplates.h file:

class ClsWithTemplatesImplementation; // forward declaration
  // definition included in the ClsWithNoTemplates.cpp file
  // this class will have a ComplicatedTemplate<abc> member, but it is only 
  // included in your ClsWithNoTemplates definition file (that is only included once)


class ClsWithNoTemplates
{
     ClsWithTemplatesImplementation * impl; // no templates mentioned anywhere here
public:
    void FunctionUsingYourMember(); // call impl->FunctionUsingYourMember() internally
};

This changes your OOP design a bit, but it's for the good: including the definition of 'ClsWithNoTemplates' is now fast and you only (pre)compile the definition of 'ClsWithNoTemplates' once.

Aditionally, if you change the implementation code, any code that included ClsWithNoTemplates.h will probably not need to be redefined.

This change should dramatically increase your partial compilation time, and it will also help in the case where your ClsWithNoTemplates is a public interface exported from a library file: since the file is not changed when you only change the implementation, your dependent client code doesn't need to be recompiled at all.