By default Xcodes performance tests are run ten times and my result is the average of those ten tests. The problem is the averaged result varies considerably each time I run it
I typically expand the Standard Deviation value to include the range I'd accept.
One obvious workaround for you is to perform the operation 5 times within the measure block you are executing. (You'll want to change the expected time.)