AI Applications in C++: How costly are virtual functions? What are the possible optimizations?

Question

In an AI application I am writing in C++,

1. there is not much numerical computation
2. there are lot of structures for which run-time polymorphism is ne

Answer 1

As already stated by the other answers, the actual overhead of a virtual function call is fairly small. It may make a difference in a tight loop where it is called millions of times per second, but it's rarely a big deal.

However, it may still have a bigger impact in that it's harder for the compiler to optimize. It can't inline the function call, because it doesn't know at compile-time which function will be called. That also makes some global optimizations harder. And how much performance does this cost you? It depends. It is usually nothing to worry about, but there are cases where it may mean a significant performance hit.

And of course it also depends on the CPU architecture. On some, it can become quite expensive.

But it's worth keeping in mind that any kind of runtime polymorphism carries more or less the same overhead. Implementing the same functionality via switch statements or similar, to select between a number of possible functions may not be cheaper.

The only reliable way to optimize this would be if you could move some of the work to compile-time. If it is possible to implement part of it as static polymorphism, some speedup may be possible.

But first, make sure you have a problem. Is the code actually too slow to be acceptable? Second, find out what makes it slow through a profiler. And third, fix it.

Answer 2

Virtual calls do not present much greater overhead over normal functions. Although, the greatest loss is that a virtual function when called polymorphically cannot be inlined. And inlining will in a lot of situations represent some real gain in performance.

Something You can do to prevent wastage of that facility in some situations is to declare the function inline virtual.

Class A {
   inline virtual int foo() {...}
};

And when you are at a point of code you are SURE about the type of the object being called, you may make an inline call that will avoid the polymorphic system and enable inlining by the compiler.

class B : public A {
     inline virtual int foo() 
     {
         //...do something different
     }

     void bar()
     {
      //logic...
      B::foo();
      // more  logic
     }
};

In this example, the call to foo() will be made non-polymorphic and bound to B implementation of foo(). But do it only when you know for sure what the instance type is, because the automatic polymorphism feature will be gone, and this is not very obvious for later code readers.

Answer 3

Static polymorphism, as some users answered here. For example, WTL uses this method. A clear explanation of the WTL implementation can be found at http://www.codeproject.com/KB/wtl/wtl4mfc1.aspx#atltemplates