How best to sum up lots of floating point numbers?

Question

Imagine you have a large array of floating point numbers, of all kinds of sizes. What is the most correct way to calculate the sum, with the least error? For example, when t

Answer 1

Well, if you don't want to sort then you could simply keep the total in a variable with a type of higher precision than the individual values (e.g. use a double to keep the sum of floats, or a "quad" to keep the sum of doubles). This will impose a performance penalty, but it might be less than the cost of sorting.

Answer 2

If your application relies on numeric processing search for an arbitrary precision arithmetic library, however I don't know if there are Python libraries of this kind. Of course, all depends on how many precision digits you want -- you can achieve good results with standard IEEE floating point if you use it with care.

Answer 3

There are many algorithms, depending on what you want. Usually they require keeping track of the partial sums. If you keep only the the sums x[k+1] - x[k], you get Kahan algorithm. If you keep track of all the partial sums (hence yielding O(n^2) algorithm), you get @dF 's answer.

Note that additionally to your problem, summing numbers of different signs is very problematic.

Now, there are simpler recipes than keeping track of all the partial sums:

• Sort the numbers before summing, sum all the negatives and the positives independantly. If you have sorted numbers, fine, otherwise you have O(n log n) algorithm. Sum by increasing magnitude.
• Sum by pairs, then pairs of pairs, etc.

Personal experience shows that you usually don't need fancier things than Kahan's method.

Answer 4

See also: Kahan summation algorithm It does not require O(n) storage but only O(1).

Answer 5

For "more precise": this recipe in the Python Cookbook has summation algorithms which keep the full precision (by keeping track of the subtotals). Code is in Python but even if you don't know Python it's clear enough to adapt to any other language.

All the details are given in this paper.