How to calculate percentage improvement in response time for performance testing

Question

How should I calculate the percentage improvement in response time.

I am getting 15306 ms response time for old code and 799 ms response for th

Answer 1

Couple of responses already answered the question correctly, but let me expand those answers with some additional thoughts and a practical example.

Percentage improvement is usually calculated as ((NEW - OLD)/OLD) * 100

Let us think about it with some practical examples:

• If I make $10,000 in my current job and get a new job that offers $12,000 then I am getting 20% increase in salary with this new job ((12000 - 10000)/10000)*100
• If Train A travels at 100 miles per hour and Train B travels at 150 miles per hour, Train B is 50% faster than Train A. Simple, right?

It gets tricky when you try to measure a metric using another metric that has an inverse relationship with the metric you are trying to measure. Let me explain what I mean by this.

Let us try the second example again now using "time taken to reach destination". Let us say Train A takes 3 hours to reach the destination and Train B takes 2 hours to reach the same destination. Train B is faster than Train A by what percentage?

If we use the same formula we used in the above examples, we get ((2-3)/3)*100, which is -33%. This simply tells that Train B takes 33% less time to reach the destination than Train A, but that is not what we are trying to determine. Right? We are trying to measure the difference in speed by percentage. If we change the formula slightly and take the absolute value, we get 33%, which may seem right, but not really. (I'll explain why in a minute)

So, what do we do? The first thing we need to do is to convert the metric we have in hand to the metric we want to measure. After that, we should be able to use the same formula. In this example, we are trying to measure difference in speed. So let us first get the speed of each train. Train A travels at 1/3 of the distance per hour. Train B travels at 1/2 distance per hour. The difference in speed in percentage then is: ((1/2 - 1/3)/1/3) * 100 = ((1/2 - 1/3)*3)*100 = (3/2 - 3/3) * 100 = 50%

Which happens to be same as ((3 - 2)/2) * 100.

In short, when the metric we are measuring and the metric we have at hand have an inverse relationship, the formula should be

((OLD - NEW)/NEW) * 100

What is wrong with the original formula? Why can't we use the original formula and conclude that train B is only 33% faster? Because it is inaccurate. The original formula always yields a result that is less than 100%. Imagine a flight reaching the same destination in 15 mins? The first formula tells that flight is 91.6% faster while the second formula tells that the flight is 1100% faster (or 11 times faster), which is more accurate.

Using this modified formula, the percentage improvement for case posted in the original question is ((15306 - 799)/799) * 100 = 1815.6%