How to calculate percentage improvement in response time for performance testing

Question

How should I calculate the percentage improvement in response time.

I am getting 15306 ms response time for old code and 799 ms response for th

Answer 1

The formula for finding the percentage of reduction is:

P = a/b × 100

Where P is the percentage of reduction, a is the amount of the reduction and b is the original amount that was reduced.

So to calculate a you do: old - new wichi will translate into:

P = ((OLD - NEW)/OLD)*100

Answer 2

((old time - new time) / old time) * 100
This formula will give the Percentage Decreased in New Response time.

In your case, ((15306 - 799)/ 15306) * 100 = 94.78 %

Answer 3

I think the answers above suffer from the original question not having nice round numbers and that there are 3 different ways to state the result.

Let's assume that the old time was 10 seconds and the new time is 5 seconds.

There's clearly a 50% reduction (or decrease) in the new time:

(old-new)/old x 100% = (10-5)/10 x 100% = 50%

But when you talk about an increase in performance, where a bigger increase is clearly better, you can't use the formula above. Instead, the increase in performance is 100%:

(old-new)/new x 100% = (10-5)/5 x 100% = 100%

The 5 second time is 2x faster than the 10 second time. Said a different way, you can do the task twice (2x) now for every time you used to be able to do it.

old/new = 10/5 = 2.0

---

So now let's consider the original question

The old time was 15306 ms and the new time is 799 ms.

There is a 94.7% reduction in time.

(old-new)/old x 100% = (15306-799)/15306 x 100% = 94.7%

There is a 1816% increase in performance:

(old-new)/new x 100% = (15306-799)/799 x 100% = 1815.6%

Your new time is 19x faster:

old/new = 15306/799 = 19.16

Answer 4

your code's runtime is 94.78% shorter/improved/decreased:

(new - old) / old x 100%
(799 - 15306) / 15306 x 100% =~ -94.78% (minus represents decrease)

your code is 1816% faster:

(old - new) / new x 100%
(15306 - 799) / 799 x 100% =~ 1816%

Answer 5

Couple of responses already answered the question correctly, but let me expand those answers with some additional thoughts and a practical example.

Percentage improvement is usually calculated as ((NEW - OLD)/OLD) * 100

Let us think about it with some practical examples:

• If I make $10,000 in my current job and get a new job that offers $12,000 then I am getting 20% increase in salary with this new job ((12000 - 10000)/10000)*100
• If Train A travels at 100 miles per hour and Train B travels at 150 miles per hour, Train B is 50% faster than Train A. Simple, right?

It gets tricky when you try to measure a metric using another metric that has an inverse relationship with the metric you are trying to measure. Let me explain what I mean by this.

Let us try the second example again now using "time taken to reach destination". Let us say Train A takes 3 hours to reach the destination and Train B takes 2 hours to reach the same destination. Train B is faster than Train A by what percentage?

If we use the same formula we used in the above examples, we get ((2-3)/3)*100, which is -33%. This simply tells that Train B takes 33% less time to reach the destination than Train A, but that is not what we are trying to determine. Right? We are trying to measure the difference in speed by percentage. If we change the formula slightly and take the absolute value, we get 33%, which may seem right, but not really. (I'll explain why in a minute)

So, what do we do? The first thing we need to do is to convert the metric we have in hand to the metric we want to measure. After that, we should be able to use the same formula. In this example, we are trying to measure difference in speed. So let us first get the speed of each train. Train A travels at 1/3 of the distance per hour. Train B travels at 1/2 distance per hour. The difference in speed in percentage then is: ((1/2 - 1/3)/1/3) * 100 = ((1/2 - 1/3)*3)*100 = (3/2 - 3/3) * 100 = 50%

Which happens to be same as ((3 - 2)/2) * 100.

In short, when the metric we are measuring and the metric we have at hand have an inverse relationship, the formula should be

((OLD - NEW)/NEW) * 100

What is wrong with the original formula? Why can't we use the original formula and conclude that train B is only 33% faster? Because it is inaccurate. The original formula always yields a result that is less than 100%. Imagine a flight reaching the same destination in 15 mins? The first formula tells that flight is 91.6% faster while the second formula tells that the flight is 1100% faster (or 11 times faster), which is more accurate.

Using this modified formula, the percentage improvement for case posted in the original question is ((15306 - 799)/799) * 100 = 1815.6%

Answer 6

Actually performance is about how much can be done in the same amount of time.

So the formula is OLD/NEW - 1 In your case your performance increased by 1816% (i.e. you can do 18.16X more in the same time)

15306/799 - 1 = 1816%

Note: before you could do 1/15360, now 1/799 ...