r-future | 易学教程

Parallelization on resampling within a stacked learner (ensemble/stack of classification learners) doesn't work

阅读更多关于 Parallelization on resampling within a stacked learner (ensemble/stack of classification learners) doesn't work

问题 The below code works fine, however, I am interested to run it in parallel. I have tried different plans within future and future.apply but couldn't managed. Any help appreciated. I am running on windows OS, 8 cores. library(mlr3verse) library(future.apply) #> Warning: package 'future.apply' was built under R version 3.6.3 #> Loading required package: future #> Warning: package 'future' was built under R version 3.6.3 library(future) future::plan(multicore) tsk_clf = tsk("sonar") tsk_clf$col

R parallel makeCluster() hangs infinitely on Mac

阅读更多关于 R parallel makeCluster() hangs infinitely on Mac

问题 I came across some problem when I was trying to use parallel package in R on my Mac. Here is how the parallel package works normally. cl = makeCluster(2) # Using 2-core parallel as an example # Your parallel code stopCluster(cl) When I ran this code, the cl = makeCluster(2) hangs infinitely . I was trying to solve it but failed. I also referred to some other posts. Several potential reasons includes not enough memory, installation error, etc. They do not seems to be the problem here, as I

future waits for execution on single-core machine

阅读更多关于 future waits for execution on single-core machine

问题 I have 2 machines with identical versions of Ubuntu (16.04), R (3.4.4) and the future package (1.10.0). I am running the code below, that implements asyncronous programming tic() library(future) plan(multisession) mehAsync = function(){ future::future({ Sys.sleep(5) return('meh') }) } out = mehAsync() what = then(out,function(value){(value)}) toc() The expected behaviour is that the execution will end almost instantly, leaving what to be a pending promise. In one machine this happens, the

Internal function of R package not found when using foreach with checkpoint

阅读更多关于 Internal function of R package not found when using foreach with checkpoint

问题 This is a follow-up question of this question: How to set .libPaths (checkpoint) on workers when running parallel computation in R Based on the answer I put the following code (simplified example) into an R package called 'test1': #' @export f <- function() { `%dopar%` <- foreach::`%dopar%` doFuture::registerDoFuture() libs <- .libPaths() res <- foreach::foreach(x = 1:2) %dopar% { cat(sprintf("Initial Library paths used by worker (PID %d):\n", Sys.getpid())) cat(sprintf(" - %s\n", sQuote(

Asynchronous programming in R

阅读更多关于 Asynchronous programming in R

问题 Overview I am writing a program (in R) that makes API calls at certain designated times. The API calls take a while, but I need the timer (main loop) to continue counting while the API call is made. To do so, I need to "outsource" the API call to another CPU thread. I believe this is possible and have looked into the future and promises packages, but haven't found a solution yet. Reproducible Example Let's run a for loop that counts from 0 to 100. When the counter ( i ) gets to 50, it has to

R parallel makeCluster() hangs infinitely on Mac

阅读更多关于 R parallel makeCluster() hangs infinitely on Mac

I came across some problem when I was trying to use parallel package in R on my Mac. Here is how the parallel package works normally. cl = makeCluster(2) # Using 2-core parallel as an example # Your parallel code stopCluster(cl) When I ran this code, the cl = makeCluster(2) hangs infinitely . I was trying to solve it but failed. I also referred to some other posts. Several potential reasons includes not enough memory, installation error, etc. They do not seems to be the problem here, as I restarted sessions, reinstalled R, but the problem remained. I guess the problem is about the permission

How to set .libPaths (checkpoint) on workers when running parallel computation in R

阅读更多关于 How to set .libPaths (checkpoint) on workers when running parallel computation in R

问题 I use the checkpoint package for reproducible data analysis. Some of the computations take a long time to compute, so I want to run those in parallel. When run in parallel however the checkpoint is not set on the workers, so I get an error message "there is no package called xy" (because it is not installed in my default library directory). How can I make sure, that each worker uses the package versions in the checkpoint folder? I tried to set .libPaths in the foreach code but this does not