Deferred & Remote Function Execution in R

What would you say if you could automatically wrap your R pipeline – which consists of numerous functions and variables – into a single function? What would you say if you could do it repeatedly, with no extra effort regardless of its complexity? Would you store it to document your progress at that particular moment in time? Would you serialize and send it to a remote R session to offload computations? Well, it’s kind-of possible.

defer brings a new capability to R – packaging, as a single R function, a number of inter-connected functions and variables which exist in the current R session. It is something I found useful during fast-prototyping phases of my own work and wanted to share for quite some time now – and at this point I’m interested in seeing if it turns out useful for someone else besides me.

Here’s a simplistic example where defer is given a top-level function h() that requires two other functions (f() and g()), a variable (C) and a library call (base::mean).

C <- 10

f <- function(x) x*x + C
g <- function(y) (f(y) + 10) * f(y+3)
h <- function(z) mean(c(g(z), g(z+1), g(z+2)))

wrapper <- defer(h)
#> Found functions:
#>   g, f
#> variables:
#>   C
#> library calls:
#>   base::mean

rm(C, f, g, h)

wrapper(10)
#> [1] 29688.67

Here is another example where that same wrapper() can be run in a remote R session (via opencpu) without installing or deploying any extra code on the remote server. First, we serialize wrapper as a base64-encoded string.

library(base64enc)

buffer <- rawConnection(raw(0), 'w')
saveRDS(wrapper, buffer)
serialized <- base64enc::base64encode(rawConnectionValue(buffer))

Then, we write that string into a temporary file together with a few lines of code to deserialize and run it in a remote R session.

local_script_path <- tempfile(fileext = ".R")

cat(paste0("base64 <- '", serialized, "'\n",
           "library(base64enc)\n",
           "bytes <- rawConnection(base64enc::base64decode(base64), 'r')\n",
           "deserialized <- readRDS(bytes)\n",
           "deserialized(10)\n"),
    file = local_script_path)

Finally, we upload the file on the opencpu server and run it via base::source.

library(httr)
result <- httr::POST(public_opencpu_url,
                     body = list(file = upload_file(local_script_path)))
content(result, 'text')
#> [1] "$value\n[1] 29688.67\n\n$visible\n[1] TRUE\n\n"

And that seems like a number we’ve seen already, right?

The package is currently available only from GitHub as it’s way not ready for CRAN.