As Knuth said,
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.
The way I see it is, if you optimize something without knowing how much performance you can gain in different scenario IS a premature optimization. The goal of code should really making it easiest for human to read.