When is optimisation premature?

Question

As Knuth said,

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

Answer 1

Optimization can happen at different levels of granularity, from very high-level to very low-level:

1. Start with a good architecture, loose coupling, modularity, etc.

2. Choose the right data structures and algorithms for the problem.

3. Optimize for memory, trying to fit more code/data in the cache. The memory subsystem is 10 to 100 times slower than the CPU, and if your data gets paged to disk, it's 1000 to 10,000 times slower. Being cautious about memory consumption is more likely to provide major gains than optimizing individual instructions.

4. Within each function, make appropriate use of flow-control statements. (Move immutable expressions outside of the loop body. Put the most common value first in a switch/case, etc.)

5. Within each statement, use the most efficient expressions yielding the correct result. (Multiply vs. shift, etc)

Nit-picking about whether to use a divide expression or a shift expression isn't necessarily premature optimization. It's only premature if you do so without first optimizing the architecture, data structures, algorithms, memory footprint, and flow-control.

And of course, any optimization is premature if you don't define a goal performance threshold.

In most cases, either:

A) You can reach the goal performance threshold by performing high-level optimizations, so it's not necessary to fiddle with the expressions.

or

B) Even after performing all possible optimizations, you won't meet your goal performance threshold, and the low-level optimizations don't make enough difference in performance to justify the loss of readability.

In my experience, most optimization problems can be solved at either the architecture/design or data-structure/algorithm level. Optimizing for memory footprint is often (though not always) called for. But it's rarely necessary to optimize the flow control & expression logic. And in those cases where it actually is necessary, it's rarely sufficient.