Different summation results with Parallel.ForEach

Question

I have a foreach loop that I am parallelizing and I noticed something odd. The code looks like

double sum = 0.0;

Parallel.ForEach(myCollection         


        

Answer 1

If you think about that sum += ComplicatedFunctionas being actually composed of a bunch of operations, say:

r1 <- Load current value of sum
r2 <- ComplicatedFunction(...)
r1 <- r1 + r2

So now we randomly interleave two (or more) parallel instances of this. One thread may be holding a stale "old value" of sum which it uses to perform its computation, the result of which it writes back over top of some modified version of sum. It's a classic race condition, because some results are getting lost in a nondeterministic way based on how the interleaving is done.