mclapply | 易学教程

An error in one job contaminates others with mclapply

阅读更多关于 An error in one job contaminates others with mclapply

问题 When mclapply(X, FUN) encounters errors for some of the values of X , the errors propagate to some (but not all) of the other values of X : require(parallel) test <- function(x) if(x == 3) stop() else x mclapply(1:3, test, mc.cores = 2) #[[1]] #[1] "Error in FUN(c(1L, 3L)[[2L]], ...[cut] # #[[2]] #[1] 2 # #[[3]] #[1] "Error in FUN(c(1L, 3L)[[2L]], ... [cut] #Warning message: #In mclapply(1:3, test, mc.cores = 2) : # scheduled core 1 encountered error in user code, all values of the job will

parallel r with foreach and mclapply at the same time

阅读更多关于 parallel r with foreach and mclapply at the same time

问题 I am implementing a parallel processing system which will eventually be deployed on a cluster, but I'm having trouble working out how the various methods of parallel processing interact. I need to use a for loop to run a big block of code, which contains several large list of matrices operations. To speed this up, I want to parallelise the for loop with a foreach(), and parallelise the list operations with mclapply. example pseudocode: cl<-makeCluster(2) registerDoParallel(cl) outputs <-

mclapply vs for loops for plotting: speed and scalability focus

阅读更多关于 mclapply vs for loops for plotting: speed and scalability focus

问题 I am running a function in R that can take a long time to run as it carries out multiple commands to transform and subset some data before it pushes it into ggplot to plot. I need to run this function multiple times adjusting the arguments values. The example I will provide is a simple one...but was wondering how to speed it up? if scaled up, i.e. what is the fastest way of getting every single combination...is there a generic method of converting for loops into mclapply assuming they are

R: TermDocumentMatrix - Error while creating

阅读更多关于 R: TermDocumentMatrix - Error while creating

问题 I am trying to get twitter data and create a wordcloud but my code is giving error while creating TermDocumentMatrix. My code is as below twitter_search_data <- searchTwitter(searchString = text_to_search ,n = 500) twitter_search_text <- sapply(twitter_search_data ,function(x) x$getText()) twitter_search_corpus <- Corpus(VectorSource(twitter_search_text)) twitter_search_corpus <- tm_map(twitter_search_corpus, stripWhitespace, lazy = TRUE) twitter_search_corpus <- tm_map(twitter_search_corpus,

assembling a matrix from diagonal slices with mclapply or %dopar%, like Matrix::bandSparse

阅读更多关于 assembling a matrix from diagonal slices with mclapply or %dopar%, like Matrix::bandSparse

问题 Right now I'm working with some huge matrices in R and I need to be able to reassemble them using diagonal bands. For programming reasons (to avoid having to do n*n operations for a matrix of size n (millions of calculations), I wanted to just do 2n calculations (thousands of calculations) and thus chose to do run my function on the diagonal bands of the matrix. Now, I have the results, but need to take these matrix slices and assemble them in a way that allows me to use multiple processors.

mclapply encounters errors depending on core id?

阅读更多关于 mclapply encounters errors depending on core id?

问题 I have a set of genes for which I need to calculate some coefficients in parallel. Coefficients are calculated inside GeneTo_GeneCoeffs_filtered that takes gene name as an input and returns the list of 2 data frames. Having 100-length gene_array I ran this command with the different number of cores: 5, 6 and 7. Coeffslist=mclapply(gene_array,GeneTo_GeneCoeffs_filtered,mc.cores = no_cores) I encounter errors on different gene names depending on the number of cores assigned to mclapply .

R Checking for duplicates is painfully slow, even with mclapply

阅读更多关于 R Checking for duplicates is painfully slow, even with mclapply

问题 I've got some data involving repeated sales for a bunch of of cars with unique Ids. A car can be sold more than once. Some of the Ids are erroneous however, so I'm checking, for each Id, if the size is recorded as the same over multiple sales. If it isn't, then I know that the Id is erroneous. I'm trying to do this with the following code: library("doMC") Data <- data.frame(ID=c(15432,67325,34623,15432,67325,34623),Size=c("Big","Med","Small","Big","Med","Big")) compare <- function(v) all

Looping multiple listed data frames into a single function

阅读更多关于 Looping multiple listed data frames into a single function

问题 I am trying to execute the function varipart() from the package ade4. I am trying to use the same number dataframe from each list in the different parts of the same function. I need to pass this for each set of dataframes. ########### DATA BELOW d1 <- data.frame(y1 = c(1, 2, 3), y2 = c(4, 5, 6)) d2 <- data.frame(y1 = c(3, 2, 1), y2 = c(6, 5, 4)) d3 <- data.frame(y1 = c(2, 1, 2), y2 = c(5, 6, 4)) spec.list <- list(d1, d2, d3) d1 <- data.frame(y1 = c(20, 87, 39), y2 = c(46, 51, 8)) d2 <- data

parallel::mclapply() adds or removes bindings to the global environment. Which ones?

阅读更多关于 parallel::mclapply() adds or removes bindings to the global environment. Which ones?

Why this matters For drake , I want users to be able to execute mclapply() calls within a locked global environment. The environment is locked for the sake of reproducibility. Without locking, data analysis pipelines could invalidate themselves . Evidence that mclapply() adds or removes global bindings set.seed(0) a <- 1 # Works as expected. rnorm(1) #> [1] 1.262954 tmp <- parallel::mclapply(1:2, identity, mc.cores = 2) # No new bindings allowed. lockEnvironment(globalenv()) # With a locked environment a <- 2 # Existing bindings are not locked. b <- 2 # As expected, we cannot create new

An error in one job contaminates others with mclapply

阅读更多关于 An error in one job contaminates others with mclapply

When mclapply(X, FUN) encounters errors for some of the values of X , the errors propagate to some (but not all) of the other values of X : require(parallel) test <- function(x) if(x == 3) stop() else x mclapply(1:3, test, mc.cores = 2) #[[1]] #[1] "Error in FUN(c(1L, 3L)[[2L]], ...[cut] # #[[2]] #[1] 2 # #[[3]] #[1] "Error in FUN(c(1L, 3L)[[2L]], ... [cut] #Warning message: #In mclapply(1:3, test, mc.cores = 2) : # scheduled core 1 encountered error in user code, all values of the job will be affected How can I stop this happening? The trick is to set mc.preschedule = FALSE mclapply(1:3, test