domc

make .combine function scaleable

旧街凉风 提交于 2020-01-01 07:02:53
问题 I am trying to use foreach and am having problems making the .combine function scalable. For example, here is a simple combine function MyComb <- function(part1,part2){ xs <- c(part1$x,part2$x) ys <- c(part1$y,part2$y) return(list(xs,ys)) } When I use this function to combine a foreach statement with an iterator other than 2 it returns it incorrectly. For example this works: x = foreach(i=1:2,.combine=MyComb) %dopar% list("x"=i*2,"y"=i*3) But not this: x = foreach(i=1:3,.combine=MyComb)

package doMC NOT available for R version 3.0.0 warning in install.packages

萝らか妹 提交于 2019-12-30 03:18:09
问题 For some packages like doMC & doSMP, i get the warning & inability to library(doMC). As shown below, i have no problem with subselect thus no file/directory permission issue. Also tried repo=http://cran.us.r-project.org & others, no luck. please advise. ps: for the unexpected Japanese characters, i have no clue; i assume that is a separable issue; no response yet from support.rstudio.org/help/discussions/problems/6009-japanese-characters-show-unexpectedly > install.packages('doMC') Warning in

doMC vs doSNOW vs doSMP vs doMPI: why aren't the various parallel backends for 'foreach' functionally equivalent?

余生长醉 提交于 2019-12-28 05:16:26
问题 I've got a few test pieces of code that I've been running on various machines, always with the same results. I thought the philosophy behind the various do... packages was that they could be used interchangeably as a backend for foreach's %dopar%. Why is this not the case? For example, this code snippet works: library(plyr) library(doMC) registerDoMC() x <- data.frame(V= c("X", "Y", "X", "Y", "Z" ), Z = 1:5) ddply(x, .(V), function(df) sum(df$Z),.parallel=TRUE) While each of these code

Allow foreach workers to register and distribute sub-tasks to other workers

我与影子孤独终老i 提交于 2019-12-21 17:07:28
问题 I have an R code that involves several foreach workers to perform some tasks in parallel. I am using foreach and doMC for this purpose. I want to let each of the foreach workers recruits some new workers and distribute some parts of their code, which is parallelizable, to them. The current code looks like: require(doMC) require(foreach) registerDoMC(cores = 8) foreach (i = (1:8)) %dopar% { <<some code here>> for (j in c(1:4)) { <<some other code here>> } } I am looking for an ideal code that

the difference between doMC and doParallel in R

做~自己de王妃 提交于 2019-12-17 21:47:00
问题 What's the difference between doParallel and doMC in R concerning foreach function? doParallel supports windows, unix-like, while doMC supports unix-like only. In other words, why doParallel cannot replace doMC directly? Thank you. Update: doParallel is built on parallel , which is essentially a merger of multicore and snow and automatically uses the appropriate tool for your system. As a result, we can use doParallel to support multi systems. In other words, we can use doParallel to replace

R no longer runs in parallel

删除回忆录丶 提交于 2019-12-12 21:09:34
问题 I run (k)Ubuntu 12.04.2 and R 3.0.1 I wrote a bunch of code that used to run in parallel, but now it no longer does. Not even this runs in parallel any more: library(doMC) registerDoMC(4) Results = foreach (i = 1:1e6, .combine = "c") %dopar% { sqrt(i) } And that definitely should. What I think broke it is either the R 3.0.1 update or a -dev, -devel BLAS package I installed. (openBLAS I think) I've tried system(sprintf("taskset -p 0xffffffff %d", Sys.getpid())) as suggested elsewhere, and get

assembling a matrix from diagonal slices with mclapply or %dopar%, like Matrix::bandSparse

╄→尐↘猪︶ㄣ 提交于 2019-12-11 08:37:12
问题 Right now I'm working with some huge matrices in R and I need to be able to reassemble them using diagonal bands. For programming reasons (to avoid having to do n*n operations for a matrix of size n (millions of calculations), I wanted to just do 2n calculations (thousands of calculations) and thus chose to do run my function on the diagonal bands of the matrix. Now, I have the results, but need to take these matrix slices and assemble them in a way that allows me to use multiple processors.

Going from multi-core to multi-node in R

与世无争的帅哥 提交于 2019-12-11 03:54:53
问题 I've gotten accustomed to doing R jobs on a cluster with 32 cores per node. I am now on a cluster with 16 cores per node. I'd like to maintain (or improve) performance by using more than one node (as I had been doing) at a time. As can be seen from my dummy sell script and dummy function (below), parallelization on a single node is really easy. Is it similarly easy to extend this to multiple nodes? If so, how would I modify my scripts? R script: library(plyr) library(doMC) registerDoMC(16)

R Checking for duplicates is painfully slow, even with mclapply

时光毁灭记忆、已成空白 提交于 2019-12-10 20:01:37
问题 I've got some data involving repeated sales for a bunch of of cars with unique Ids. A car can be sold more than once. Some of the Ids are erroneous however, so I'm checking, for each Id, if the size is recorded as the same over multiple sales. If it isn't, then I know that the Id is erroneous. I'm trying to do this with the following code: library("doMC") Data <- data.frame(ID=c(15432,67325,34623,15432,67325,34623),Size=c("Big","Med","Small","Big","Med","Big")) compare <- function(v) all

tryCatch with a complicated function and plyr in R

血红的双手。 提交于 2019-12-07 23:13:10
问题 I've got a complicated, long function that I'm using to do simulations. It can generate errors, mostly having to do with random vectors ending up with equal values with zero-variance, getting fed either into PCA's or logistic regressions. I'm executing it on a cluster using doMC and plyr . I don't want to tryCatch every little thing inside of the function, because the possibilities for errors are many and the probabilities of each of them are small. How do I tryCatch each run, rather than