Constricted development with reticulate

I’ve been using the reticulate package occasionally for a while now, so I was surprised to see that it had only just been officially released.

reticulate: R interface to Python https://t.co/qVWmwoMQAP. Comprehensive set of interoperability tools including R Markdown Python engine #rstats #pydata pic.twitter.com/SuWM6Y3Pk0

— RStudio (@rstudio) March 26, 2018

It’s a brilliant piece of work, allowing python and R to coexist in the same workflow.

Another opportunity came up today to use it so I thought it might be nice to do a very quick blog post to show just how easy it is to take external python code and have it callable directly from R. In this case, [@coolbutuseless](https://twitter.com/coolbutuseless) posed a challenge on Twitter to write a fast ‘needle in a haystack’ search of a small vector inside a larger one. I looked over the existing candidates and figured some sort of Sieve of Eratosthenes-esque algorithm might have a chance (though the name eluded me entirely at the time).

My proposal was to search for the first digit using which(), and use this reduced vector of possible-matches in additional tests on the remaining parts of the ‘needle’. [@coolbutuseless](https://twitter.com/coolbutuseless) refactored my attempt allowing for arbitrary length needles and found it to do quite well against the current offerings. What he still wanted though was a Boyer–Moore string search algorithm implementation. This is apparently what GNU grep uses, so it’s probably pretty okay.

That algorithm is pretty clever about how it goes about the search, starting in a similar way to what I did (the sieve approach was apparently the leading string match method prior to Boyer-Moore). It’s much more complicated though, so I wasn’t about to write one of those myself in R. Nowadays, people think of C/C++ when there’s functionality they want to grab from elsewhere. There’s a C implementation on the Wikipedia site, so that seems like a nice place to start. I saved the text to a new boyermoore.c file and ran

R CMD SHLIB boyermoore.c

from a terminal to compile it into boyermore.so. This could then be loaded into R with dyn.load("boyermore.so") and in theory called with .C("boyer_moore", <something>, <something>). I tried a couple of <something>s (which weren’t pointers) and promptly crashed RStudio.

The python implementation is also listed on Wikipedia, so I figured that’s another route to try. I saved the text to a new boyermoor.py file (also embedded below) and started about loading the functions from R. This is actually much simpler than for C:

library(reticulate)
bm <- py_run_file("boyermoor.py")

This executes the python file and creates a new named list with each exported python function as an element. How easy is that!?! Calling the function would be as easy as

bm$string_search(needle, haystack)

Not quite that easy of course… The implementation assumes that both the ‘needle’ and the ‘haystack’ are text, not numbers. To solve this, I converted my numbers (in the range 0 to 12) to letters using the built-in LETTERS vhat it worked as expected, a benchmark test showed that it was nowhere near as fast as my R approach. I can’t say this is due to the algorithm itself, which should be fairly fast, but probably has more to do with the fact that I’m using two different languages.

The entire call from R looks pretty neat and tidy

{{% gist jonocarroll d658b5ccf33aaef150b6b36f055d2d6d %}}

despite a lot of python code in the background

I’d certainly recommend having reticulate in your arsenal next time you need to attack a problem using python from within R. There’s a whole heap of useful ways to interact between R and python with this including importing python modules and calling python scripts, etc…

As a side-note: keep an eye on the ergo project to connect the go language in just as easily.

< summary> devtools::session_info()

## ─ Session info ──────────────────────────────────────────────────────────
##  setting  value                       
##  version  R version 3.5.2 (2018-12-20)
##  os       Pop!_OS 19.04               
##  system   x86_64, linux-gnu           
##  ui       X11                         
##  language en_AU:en                    
##  collate  en_AU.UTF-8                 
##  ctype    en_AU.UTF-8                 
##  tz       Australia/Adelaide          
##  date     2019-08-13                  
## 
## ─ Packages ──────────────────────────────────────────────────────────────
##  package     * version date       lib source                           
##  assertthat    0.2.1   2019-03-21 [1] CRAN (R 3.5.2)                   
##  backports     1.1.4   2019-04-10 [1] CRAN (R 3.5.2)                   
##  blogdown      0.14.1  2019-08-11 [1] Github (rstudio/blogdown@be4e91c)
##  bookdown      0.12    2019-07-11 [1] CRAN (R 3.5.2)                   
##  callr         3.3.1   2019-07-18 [1] CRAN (R 3.5.2)                   
##  cli           1.1.0   2019-03-19 [1] CRAN (R 3.5.2)                   
##  crayon        1.3.4   2017-09-16 [1] CRAN (R 3.5.1)                   
##  desc          1.2.0   2018-05-01 [1] CRAN (R 3.5.1)                   
##  devtools      2.1.0   2019-07-06 [1] CRAN (R 3.5.2)                   
##  digest        0.6.20  2019-07-04 [1] CRAN (R 3.5.2)                   
##  evaluate      0.14    2019-05-28 [1] CRAN (R 3.5.2)                   
##  fs            1.3.1   2019-05-06 [1] CRAN (R 3.5.2)                   
##  glue          1.3.1   2019-03-12 [1] CRAN (R 3.5.2)                   
##  htmltools     0.3.6   2017-04-28 [1] CRAN (R 3.5.1)                   
##  jsonlite      1.6     2018-12-07 [1] CRAN (R 3.5.1)                   
##  knitr         1.24    2019-08-08 [1] CRAN (R 3.5.2)                   
##  lattice       0.20-38 2018-11-04 [1] CRAN (R 3.5.1)                   
##  magrittr      1.5     2014-11-22 [1] CRAN (R 3.5.1)                   
##  Matrix        1.2-17  2019-03-22 [1] CRAN (R 3.5.2)                   
##  memoise       1.1.0   2017-04-21 [1] CRAN (R 3.5.1)                   
##  pkgbuild      1.0.4   2019-08-05 [1] CRAN (R 3.5.2)                   
##  pkgload       1.0.2   2018-10-29 [1] CRAN (R 3.5.1)                   
##  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.5.1)                   
##  processx      3.4.1   2019-07-18 [1] CRAN (R 3.5.2)                   
##  ps            1.3.0   2018-12-21 [1] CRAN (R 3.5.1)                   
##  R6            2.4.0   2019-02-14 [1] CRAN (R 3.5.1)                   
##  Rcpp          1.0.2   2019-07-25 [1] CRAN (R 3.5.2)                   
##  remotes       2.1.0   2019-06-24 [1] CRAN (R 3.5.2)                   
##  reticulate  * 1.13    2019-07-24 [1] CRAN (R 3.5.2)                   
##  rlang         0.4.0   2019-06-25 [1] CRAN (R 3.5.2)                   
##  rmarkdown     1.14    2019-07-12 [1] CRAN (R 3.5.2)                   
##  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.5.1)                   
##  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.5.1)                   
##  stringi       1.4.3   2019-03-12 [1] CRAN (R 3.5.2)                   
##  stringr       1.4.0   2019-02-10 [1] CRAN (R 3.5.1)                   
##  testthat      2.2.1   2019-07-25 [1] CRAN (R 3.5.2)                   
##  usethis       1.5.1   2019-07-04 [1] CRAN (R 3.5.2)                   
##  withr         2.1.2   2018-03-15 [1] CRAN (R 3.5.1)                   
##  xfun          0.8     2019-06-25 [1] CRAN (R 3.5.2)                   
##  yaml          2.2.0   2018-07-25 [1] CRAN (R 3.5.1)                   
## 
## [1] /home/jono/R/x86_64-pc-linux-gnu-library/3.5
## [2] /usr/local/lib/R/site-library
## [3] /usr/lib/R/site-library
## [4] /usr/lib/R/library