Error in unserialize(socklist[[n]]) : error reading from connection on Unix

Question

I have tried running the following code on a Unix machine with 20 CPU, using R foreach, parallel, doParallel, and party p

Answer 1

The error:

Error in unserialize(socklist[[n]]) : error reading from connection

means that the master process got an error when calling unserialize to read from the socket connection to one of the workers. That probably means that the corresponding worker died, thus dropping its end of the socket connection. Unfortunately, it may have died for any number of reasons, many of which are very system specific.

You can usually figure out why the worker died by using the makeCluster "outfile" option so that the error message generated by the worker isn't thrown away. I usually recommend using outfile="" as described in this answer. Note that the "outfile" option works the same in both the snow and parallel packages.

You could also verify that your foreach loop works correctly when executed sequentially by registering the sequential backend:

registerDoSEQ()

If you're lucky, the foreach loop will fail when executed sequentially, since it's usually easier to figure out what is going wrong.