How to do “performance-based” (benchmark) unit testing in Python

Question

Let\'s say that I\'ve got my code base to as high a degree of unit test coverage as makes sense. (Beyond a certain point, increasing coverage doesn\'t have a good ROI.)

Answer 1

You will want to do performance testing at a system level if possible - test your application as a whole, in context, with data and behaviour as close to production use as possible.

This is not easy, and it will be even harder to automate it and get consistent results.

Moreover, you can't use a VM for performance testing (unless your production environment runs in VMs, and even then, you'd need to run the VM on a host with nothing else on).

When you say doing performance unit-testing, that may be valuable, but only if it is being used to diagnose a problem which really exists at a system level (not just in the developer's head).

Also, performance of units in unit testing sometimes fails to reflect their performance in-context, so it may not be useful at all.