Option “cores” from package doParallel useless on Windows?

问题

On a Linux computer, following doParallel's vignette, I use doParallel::registerDoParallel() and then I use options(cores = N) where N is the number of cores I want to use with foreach.

I can verify with foreach::getDoParWorkers() that when I change the option cores, it automatically changes the number of cores used by foreach.

Yet, on Windows 10 (latest versions of R and packages), this option doesn't seem to have any effect as changing its value doesn't change the value of foreach::getDoParWorkers() (which is initialized at 3 when calling doParallel::registerDoParallel()).

Reproducible example:

doParallel::registerDoParallel()
options(cores = 1)
foreach::getDoParWorkers()
options(cores = 2)
foreach::getDoParWorkers()
options(cores = 4)
foreach::getDoParWorkers()

Is it a bug? Won't it work on Windows?

Edit: I know how to register parallel backends differently. The goal is to use doParallel::registerDoParallel() registering once (at the loading of my package) and then use an option to change the number of cores used. This is why I want it to work also on Windows.

回答1:

The answer from the maintainer of package doParallel, Rich Calaway:

Windows does not support forking, which is what the parallel (and doParallel) packages use the “cores” argument for. So, on Windows, all “cores” arguments are set to 1. To use multiple cores on Windows with doParallel, use makeCluster to create a multiple worker cluster cl, then registerDoParallel(cl).

So this isn't a bug, but a non-Windows feature, which is a pity.

回答2:

You will have to configure it by using both parallel and doParallel package.

• first create cluster

• register do parallel with that

  library(parallel)
  library(doParallel)
  cluster <- makeCluster(3)
  registerDoParallel(cluster)
  foreach::getDoParWorkers()
  [1] 3
  

回答3:

What about to stop the cluster and create a new process??

For example:

stops<-function(cl,n){
stopCluster(cl)
cl<-makeCluster(n)
doParallel::registerDoParallel(cl)

}

来源：https://stackoverflow.com/questions/45819337/option-cores-from-package-doparallel-useless-on-windows