The benefit

A makefile ensures that all of your work gets done, but only when it needs to and that each step has the raw material it needs to work. Identify what output you expect to see and how to generate that output and the make utility will go to work. If the output is already there, it will skip to the next thing which needs to get done. Like a 12-bar blues, very simple in concept, but easy to extend to all sorts of complex derivations. Here’s my approach:

How does it work?

I don’t really know. Sorry. I’ve had a go with the GNU documentation and it’s pretty overwhelming. I took Christopher’s basic example and modified it for my purposes. Boiling it down to basic principles, know several things:

1. make operates by building “targets”. Once the full set of targets is built, make is finished.
2. Each target has (probably) a “prerequisite”, which it needs in order to get built. The prerequisite may also be (and often is) a target itself.
3. The rule for building the target is called a “recipe” and is typically a shell command.
4. make makes liberal use of variables and wildcards. A variable is often written in all caps. To refer to it, enclose it within parentheses and precede it with a dollar sign, e.g. $(MY_VARIABLE).
5. You can also include a “clean” step, which will wipe out all the targets. This will ensure that everything gets rebuilt.

I took a few minutes to straighten out my just-for-fun Baseball repository to align it with my current preferred workflow. Here’s what I did: first, ensure that the directory structure stuck to the gather->process->analyze-> present flow. In this case, that just meant a bit of tidying in my data directory. Second, copy the boilerplate makefile from my gist. Finally, alter the “Project Options” section of the “Tools/Project Options” in RStudio to ensure that the build tool moves from “none” to “Makefile”. That’s it. Let me say that again. That’s it.

NOTE: Please observe the copyright and limited use license at the Lahman site.

Let’s walk through the makefile. The first thing we do is establish where the root directories are. Note the use of variable substition in defining the data directory.

    RDIR = .
    DATA_DIR = $(RDIR)/data

Next, we’ll use wildcards to establish all of the .Rmd files in each of our four steps as prerequisites. I’ll just show this for the “gather” step. In the second step, the wildcard command, will pull every .Rmd file. The third line will perform a substition to construct a list of targets.

    GATHER_DIR = $(DATA_DIR)/gather
    GATHER_SOURCE = $(wildcard $(GATHER_DIR)/*.Rmd)
    GATHER_OUT = $(GATHER_SOURCE:.Rmd=.docx)

The step with the target of “all” is the key. “all” has prerequisites that are targets of each of the four steps listed above. Just before defining our ultimate target, we define a variable which will act as the “recipe” for each of the steps. The $< will be substituted with the name of our various .Rmd files.

    KNIT = Rscript -e "require(rmarkdown); render('$<')"
    
    all: $(GATHER_OUT) $(PROCESS_OUT) $(ANALYSIS_OUT) $(PRESENTATION_OUT)

We’re ready to roll. Again, we’ll just show the “gather” step. This will use another form of wildcard which will associate a .Rmd file with its target. That prerequisite name will be fed into the KNIT variable we defined earlier.

    $(GATHER_DIR)/%.docx:$(GATHER_DIR)/%.Rmd
        $(KNIT) 

Within RStudio, you can just hit CTRL-SHIFT-B to execute make. You’ll see the markdown engine zip through its files and eventually, you’ll see a pile of documentation produced. If everything went well, the next time you execute make it will tell you that nothing needs to be done. Change a .Rmd file in the processing step, though, and it will recreate that file and reperform all of the analysis. Of course, it’s possible to define things in such a way that not all of the analysis, or all of the processing, or whatever gets done if something changes upstream. I tend to customize this basic makefile to be a bit more fine tuned. However, I’ll always keep these steps in. This will give me an extra margin of safety to make sure that I’ve not ignored a critical dependency.

Conclusions

Below is a list of things that are awesome:

• Makesfiles. GNU/Linux are like the age of enlightment and the invention of moveable type combined.
• Christopher Gandrud’s book. Seriously, buy it.
• RStudio. I think everyone has gotten the memo on this, but it’s always nice to repeat. Syntax highlighting for R, SQL and shell? (And HTML and Python and C++ and CSS and JavaScript?) Single keystroke or GUI to clean or make my project? Yes, please!
• Yhui Xie. I’ve not read his book yet, but his stuff is amazing and knitr is tremendous.

## R version 3.1.3 (2015-03-09)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 14.04.2 LTS
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] knitr_1.6        RWordPress_0.2-3
## 
## loaded via a namespace (and not attached):
##  [1] digest_0.6.4    evaluate_0.5.5  formatR_0.10    htmltools_0.2.6
##  [5] RCurl_1.95-4.1  rmarkdown_0.5.1 stringr_0.6.2   tools_3.1.3    
##  [9] XML_3.98-1.1    XMLRPC_0.3-0    yaml_2.1.13