Is there way to track progress on a mclapply?

Question

I love the setting .progress = \'text\' in plyr\'s llply. However, it causes my much anxiety to not know how far along an mclapp

Answer 1

The pbapply package has implemented this for the general case (i.e. on Unix-like and Windows, also works with RStudio). Both pblapply and pbsapply have a cl argument. From the documentation:

Parallel processing can be enabled through the cl argument. parLapply is called when cl is a ’cluster’ object, mclapply is called when cl is an integer. Showing the progress bar increases the communication overhead between the main process and nodes / child processes compared to the parallel equivalents of the functions without the progress bar. The functions fall back to their original equivalents when the progress bar is disabled (i.e. getOption("pboptions")$type == "none" dopb() is FALSE). This is the default when interactive() if FALSE (i.e. called from command line R script).

If one does not supply cl (or passes NULL) the default non-parallel lapply is used, also including a progress bar.