covr: A Victory for Open Source

Aimee Gott, Mango Solutions

As an academic statistician, my life before I joined Mango was one in which the concepts of “proper programming” didn’t exist. I was one of those people who wrote ad-hoc tests when I changed code and then spent hours re-writing those ad-hoc tests when I made more changes a week later. Then I joined Mango and I am pleased to say I was introduced to concepts that would change the way I wrote code forever. Why had nobody told me about formalised unit testing before!

But still there were some things that were not so great. How did we really know if we had tested everything? Short of going through the tests by hand to make sure every branch of code was hit it wasn’t really possible. So three years ago Mango started to think about how we could handle the situation. Spurred by a number of projects we were working on at the time one of our team, Tom Taverner, started to put together a tool to help us out.

Inspired by similar tools in more traditional programming languages such as C and Java Tom began developing the testCoverage package using features that had just been put into the latest version of R, allowing you to more easily parse code. Essentially the package places unique markers in your code and then ran your unit tests. A record was kept of when each of the markers were hit and you were returned a percentage test coverage as well as an interactive visualisation of which lines of code were being executed by your tests. For an expanded description of the functionality see Andy Nicholls’ presentation from UseR! 2014.

It was a game changer for me (and for Mango more widely). It really helped us to be certain that we were catching all of the branches of our code (for example every option in an if/else) rather than just excessively testing the same elements over and over. It also became one of the tools that we used as part of ValidR for accessing the validity of a given R package.

Unfortunately we didn’t shout about it very much and, more importantly, we didn’t immediately open source the package.  Even after open-sourcing the package, much of the development continued behind closed doors with the R community receiving period releases at best.  It wasn’t easy for the community to see what we were up to or where the package was going.  Naturally others in the open source community continued to try and solve the same problem. A variety of packages made their way into the community, among them covr. Whilst we at Mango initially continued to use testCoverage most developers adopted covr. Ironically covr uses much the same approach as testCoverage but it’s a simple to use package that includes its own shiny app, providing the visual display of the coverage of your tests.

Having compared covr and testCoverage across a number of projects we’ve concluded that although there are functional pros and cons of each the popularity of covr is now such that we have decided to discontinue development of the testCoverage package.  For us it’s disappointing to have let a good idea fizzle out but it’s safe to say that we’ve learnt our lesson! Our MangoTheCat GitHub account is more active than ever with a number of tools and packages used internally by Mango making their way into the community, several of which are also on now on CRAN.

One in particular is the package goodpractice. This package is designed to help you out by giving you advice on good practice when developing packages. This might be functions that you shouldn’t use, syntax you should avoid or code complexity. It’s currently a work in progress but, crucially, it is now available on GitHub to take a look at https://github.com/MangoTheCat.  If you’re interested in what else we have planned for the R community then look out for Mango at upcoming events where we will be talking goodpractice (for example Gabor Csardi, User! 2016) and other open source tools. To learn more on package building, unit testing and other R development best practices you might also want to join us at EARL for our workshop on formal package development.