Site icon R-bloggers

progressr 0.10.1: Plyr Now Supports Progress Updates also in Parallel

[This article was first published on JottR on R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

progressr 0.10.1 is on CRAN. I dedicate this release to all plyr users and developers out there.

The progressr package provides a minimal API for reporting progress updates in R. The design is to separate the representation of progress updates from how they are presented. What type of progress to signal is controlled by the developer. How these progress updates are rendered is controlled by the end user. For instance, some users may prefer visual feedback, such as a horizontal progress bar in the terminal, whereas others may prefer auditory feedback. The progressr package works also when processing R in parallel or distributed using the future framework.

plyr + future + progressr ⇒ parallel progress reporting

The major update in this release, is that plyr (>= 1.8.7) now has built-in support for the progressr package when running in parallel. For example,

library(plyr)

## Parallelize on the local machine
future::plan("multisession")
doFuture::registerDoFuture()

library(progressr)
handlers(global = TRUE)

y <- llply(1:100, function(x) {
  Sys.sleep(1)
  sqrt(x)
}, .progress = "progressr", .parallel = TRUE)

#>   |============                                  |  28%

Previously, plyr only had built-in support for progress reporting when running sequentially. Note that the progressr is the only package that supports progress reporting when using .parallel = TRUE in plyr.

Also, whenever using progressr, the user has plenty of options for where and how progress is reported. For example, handlers("rstudio") uses the progress bar in the RStudio job interface, handlers("progress") uses terminal progress bars of the progress package, and handlers("beep") reports on progress using sounds. It’s also possible to report progress in the Shiny. See my blog post ‘progressr 0.8.0 – RStudio’s Progress Bar, Shiny Progress Updates, and Absolute Progress’ for more information.

There’s actually a better way

I actually recommend another way for reporting on progress with plyr map-reduce functions, which is more in line with the design philosophy of progressr:

The developer is responsible for providing progress updates, but it’s only the end user who decides if, when, and how progress should be presented. No exceptions will be allowed.

Please see Section ‘plyr::llply(…, .parallel = TRUE) with doFuture’ in the ‘progressr: An Introduction’ vignette for this alternative approach, which has worked for long time already. But, of course, adding .progress = "progressr" to your already existing plyr .parallel = TRUE code is as simple as it gets.

Now, make some progress!

Other posts on progress reporting

Links

To leave a comment for the author, please follow the link and comment on their blog: JottR on R.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.