How to do “performance-based” (benchmark) unit testing in Python

Question

Let\'s say that I\'ve got my code base to as high a degree of unit test coverage as makes sense. (Beyond a certain point, increasing coverage doesn\'t have a good ROI.)

Answer 1

MarkR is right... doing real-world performance testing is key, and may be somewhat dodgey in unit tests. Having said that, have a look at the cProfile module in the standard library. It will at least be useful for giving you a relative sense from commit-to-commit of how fast things are running, and you can run it within a unit test, though of course you'll get results in the details that include the overhead of the unit test framework itself.

In all, though, if your objective is zero-tolerance, you'll need something much more robust than this... cProfile in a unit test won't cut it at all, and may be misleading.