Getting Started with JAGS, rjags, and Bayesian Modelling

This post provides links to various resources on getting started with Bayesian modelling using JAGS and R. It discusses: (1) what is JAGS; (2) why you might want to perform Bayesian modelling using JAGS; (3) how to install JAGS; (4) where to find further information on JAGS; (5) where to find examples of JAGS scripts in action; (6) where to ask questions; and (7) some interesting psychological applications of Bayesian modelling.

What is JAGS?

JAGS stands for Just Another Gibbs Sampler. To quote the program author, Martyn Plummer, “It is a program for analysis of Bayesian hierarchical models using Markov Chain Monte Carlo (MCMC) simulation…” It uses a dialect of the BUGS language, similar but a little different to OpenBUGS and WinBUGS.

Why JAGS?

The question of why you might want to use JAGS can be approached in several different ways:

• Why Bayesian rather than Null Hypothesis Significance Testing (NHST) approaches?

  • To quote John D. Cook quoting Anthony O’Hagan, the benefits of “the bayesian approach are that it is 1. fundamentally sound, 2. very flexible, 3. produces clear and direct inferences, and 4. makes use of all available information.” (see John’s blog post for elaboration)
  • John K. Kruschke made a similar argument in an Open Letter extolling the benefits of the bayesian approach summarised as: “(1) Scientific disciplines from astronomy to zoology are moving to Bayesian data analysis. We should be leaders of the move, not followers. (2) Modern Bayesian methods provide richer information, with greater flexibility and broader applicability than 20th century methods. Bayesian methods are intellectually coherent and intuitive. Bayesian analyses are readily computed with modern software and hardware. (3) Null-hypothesis significance testing (NHST), with its reliance on p values, has many problems. There is little reason to persist with NHST now that Bayesian methods are accessible to everyone.”

• Why JAGS/BUGS rather than coding in a low-level language?

  • It’s simpler; for models that BUGS can handle, BUGS can shield you from some of the thorny details related to numeric integration.
  • There are simple interfaces with R.

• Why JAGS rather than WinBUGS or OpenBUGS?

  • I’m using JAGS because it works well on Ubuntu. WinBUGS is broadly Windows specific, although I’ve read that it may work with the emulation software Wine.
  • JAGS interfaces well with R. I’m comfortable writing scripts. Thus, I don’t personally see the benefits of using a dedicated GUI like WinBUGS. I can leverage what I know about R.
  • However, ultimately converting code between different flavours of BUGS is not that difficult.
  • For further discussion of the issue, see this r-help discussion and this discussion on CrossValidated.

http://stats.stackexchange.com/questions/9202/openbugs-vs-jags

More than anything I found that JAGS provided a useful entry point into the world of Bayesian modelling. This in turn appealed to me for several reasons:

1. Even when I perform analyses using an NHST approach I often intuitively think of empirical research questions in terms of probability densities on a parameter of interest that changes as empirical and theoretical evidence is accumulated. See for example Thompson’s (2002) concept of meta-analytic thinking. Bayesian analysis provides tools for formalising this orientation.
2. More broadly, I appreciate the explicitness that a Bayesian approach requires and encourages. E.g., specifying the distribution of the error term, specifying a prior, specifying the distribution of parameters in a mixed effects model, and so on.
3. There are several modelling challenges that I’m currently working through where a Bayesian approach offers substantial flexibility and applicability. In particular, I’m interested in modelling individual differences in the effect of practice on strategy use and task performance and then relating these individual differences to factors like intelligence, prior experience, and personality.

JAGS Installation

JAGS runs on Linux, Mac, and Windows. I run JAGS on Ubuntu through an interface with R called rjags.

The following sets out a basic installation process:

1. If necessary Download and install R and potentially a user interface to R like R Studio (see here for tips on getting started with R).
2. Download and install JAGS as per operating system requriements.
3. Install additional R packages: e.g., in R install.packages("rjags") . In particular, I use the packages rjags to interface with JAGS and coda to process MCMC output.

Information on JAGS

• The manual for different versions of JAGS is located here. e.g., the pdf of the manual for 3.1.0. Several particularly relevant sections include:

  • the list of supported distributions and how they are parameterised. This is often important given that the code looks similar to R but often uses different parameterisation (e.g., precision is used instead of standard deviation for a normal distribution).
  • It summarises differences between WinBUGS and JAGS.
  • It sets out available functions and operators.

• The rjags help pdf for information about how to interface with JAGS from R.

• Martin Plummer has a blog called JAGS NEWS
• The Bayesian Task View on CRAN lists and briefly describes the many R packages related to Bayesian statistics.
• Lunn and colleagues have a 2009 article called The BUGS project: Evolution, critique and future directions. It provides a useful historical perspective on the broader BUGS project, although it does not mention much about JAGS specifically.

http://cran.r-project.org/web/views/Bayesian.html http://www.stat.columbia.edu/~gelman/bayescomputation/lunnbugswithcomments.pdf

Examples JAGS Scripts

I find it easier to pick up a new language by playing with examples. The following provides links to example JAGS code, often with accompanying explanations:

• Justin Esarey
  • An entire course on Bayesian Statistics with examples in R and JAGS. It includes 10 lectures and each lecture lasts around 2 hours. The content is designed for a social science audience and it includes a syllabus linking with Simon Jackman’s text. The videos are linked from above or available direclty on YouTube

• John Myles White

  • A course on statistical models that is under development with JAGS scripts on github
  • A model of Cannabalt scores using a gamma distribution
  • Simple introductory examples of fitting a normal distribution, linear regression, and logistic regression
  • A follow-up post demonstrating the use of the coda package with rjags to perform MCMC diagnostics.

• John K. Kruschke

  • John Krushke wrote a book called Doing Bayesian Data Analysis: A Tutorial with R and BUGS. It’s an excellent entry point into the world of Bayesian statistics for the social and behavioural scientist who has reasonable quantiative training, but is not necessarily ready to absorb the kinds of books that are used in graduate-level statistics courses.
  • The book has a website that provides all the examples used in the book all the examples used in the book. See this blog post for a link to the zip file containing the JAGS code.

• BUGS Project

  • BUGS is well known for the large set of examples that accompany the project.
  • The PDF providing documentation for Volume 1 and 2 of the examples is available here.
  • You can see the JAGS code used to run these examples here.

• Patrick J Mineault

  • An example from Gelman et al examining the effect of training programs on SAT scores

• Miguel Lobo

  • A short tutorial

• Simon Jackman

  • Simon Jackman wrote the book Bayesian Analysis for the Social Sciences that has accompanying JAGS code.
  • The book’s website has several useful resources including example papers using Bayesian methods.
  • An associated course that uses the book as a text book has slides and many examples of using and R and JAGS.

• Johannes Karreth

  • A course on applied bayesian modelling with examples of data, and code using the R2jags interface.

• Myself

  • I also plan to post a few examples in upcoming blog posts. I typically will share the code for these on my github account: jeromyanglim. If you are reading this through syndication you may wish to subscribe to the RSS feed of the source blog jeromyanglim.blogspot.com.

More broadly, examples and tutorials designed for WinBUGS can generally be adapted to be useful for JAGS. So for example, you can explore these WinBUGS examples:

• Michael Lee and Eric-Jan Wagemakers have a free online book called A Course in Bayesian Graphical Modeling for Cognitive Science: see PDF and website.
• The website for the book Markov Chain Monte Carlo has several WinBUGS examples.
• There is an extensive list of BUGS resources on the BUGS project website.

Asking questions

There are several places to ask questions about JAGS, R, and Bayesian statistics.

• JAGS, BUGS, and bayesian questions on stats.stackexchange.com (aka CrossValidated).
• JAGS discussion forum
• There’s also a BUGS discussion list

In general, I prefer the Stack Exchange model for asking and answering questions on the internet, although the most important issue is typically where the experts are located.

Interesting Psychological Applications of Bayesian Modelling

If you want to see some examples of Bayesian modelling applied to psychological data, I found the following articles quite interesting. PDFs are available online.

• Shiffrin, Lee, Kim, and Wagenmakers (2008, PDF) present a tutorial on hierarchical bayesian methods in the context of cognitive science.
• Michael Lee (2011, PDF) in Journal of Mathematical Psychology discusses the benefits of hiearchical Bayesian methods to modelling psychological data and provides several example applications.
• Lee Averell and Andrew Heathcote (2010, PDF) in Journal of Mathematical Psychology analyse individual differences in the forgetting curve using a hierarchical Bayesian approach.

If you know of any other interesting JAGS resources or have any comments about my choice of software for Bayesian data analysis, feel free to post a comment.