Parfor in MATLAB Problem

Question

Why can\'t I use the parfor in this piece of code?

parfor i=1:r

    for j=1:N/r

        xr(j + (N/r) * (i-1)) = x(i + r * (j-1));

    end

end


        

Answer 1

The issue here is that of improper indexing of the sliced array. parfor loops are run asynchronously, meaning the order in which each iteration is executed is random. From the documentation:

MATLAB workers evaluate iterations in no particular order, and independently of each other. Because each iteration is independent, there is no guarantee that the iterations are synchronized in any way, nor is there any need for this.

You can easily verify the above statement by typing the following in the command line:

parfor i=1:100
    i
end

You'll see that the ordering is arbitrary. Hence if you split a parallel job between different workers, one worker has no way of telling if a different iteration has finished or not. Hence, your variable indexing cannot depend on past/future values of the iterator.

Let me demonstrate this with a simple example. Consider the Fibonacci series 1,1,2,3,5,8,.... You can generate the first 10 terms of the series easily (in a naïve for loop) as:

f=zeros(1,10);
f(1:2)=1;
for i=3:10
    f(i)=f(i-1)+f(i-2);
end

Now let's do the same with a parfor loop.

f=zeros(1,10);
f(1:2)=1;
parfor i=3:10
    f(i)=f(i-1)+f(i-2);
end

??? Error: The variable f in a parfor cannot be classified. See Parallel for Loops in MATLAB, "Overview"

But why does this give an error?

I've shown that iterations are executed in an arbitrary order. So let's say that a worker gets the loop index i=7 and the expression f(i)=f(i-1)+f(i-2);. It is now supposed to execute the expression and return the results to the master node. Now has iteration i=6 finished? Is the value stored in f(6) reliable? What about f(5)? Do you see what I'm getting at? Supposing f(5) and f(6) are not done, then you'll incorrectly calculate that the 7th term in the Fibonnaci series is 0!

Since MATLAB has no way of telling if your calculation can be guaranteed to run correctly and reproduce the same result each time, such ambiguous assignments are explicitly disallowed.