(Guest post by Achim Zeileis) Development of the R package exams for automatic generation of (statistical) exams in R started in 2006 and version 1 was published in JSS by Grün and Zeileis (2009). It was based on standalone Sweave exercises, that can be combined into exams, and then rendered into different kinds of PDF output (exams, solutions, self-study materials, etc.). Now, a major revision of the package has been released that extends the capabilities and adds support for learning management systems. It is still based on the same type of Sweave files for each exercise but can also render them into output formats like HTML (with various options for displaying mathematical content) and XML specifications for online exams in learning management systems such as Moodle or OLAT. Supplementary files such as graphics or data are handled automatically. Here, I give a brief overview of the new capabilities. A detailed discussion is in the working paper by Zeileis, Umlauf, and Leisch (2012) that is also contained in the package as a vignette.

The basic idea…

…is to have standard standalone Sweave files for each exercise. These typically comprise (1) R code chunks (as usual within <<>>= and @) for random data generation. (2) Question and solution descriptions contained in LaTeX environments of corresponding names, typically with \Sexpr{} for including data for the question. (3) Metainformation about the exercise type (numeric, multiple choice, …), its correct solution etc. Then many different replications of (collections of) such an exercise can be easily created by repeating the following four steps: Weaving the exercise (i.e., executing the R code for random data generation and embedding that into the text), reading the resulting LaTeX into R, transforming the LaTeX to another format such as HTML (if necessary), writing the corresponding output files, e.g., in PDF, HTML, XML, … The package provides convenience interfaces exams2pdf(), exams2html(), exams2moodle(), and exams2qti12() (for OLAT) that directly carry out all of these steps but the package also provides the underlying building blocks such that new interfaces can be easily created.

A simple example

To get a quick impression of the look and feel of the package, consider the following simple example: computation of a t statistic from given mean, variance, and sample size. The corresponding Sweave template tstat.Rnw is provided within the package. After installing the package, you can produce random versions of this exercise in PDF and HTML by running the following code:

library("exams")
exams2pdf("tstat.Rnw")
exams2html("tstat.Rnw")

The former needs a working LaTeX installation and then pops up a PDF with the exercise. The latter does not need LaTeX but just the R package tth (interfacing Ian Hutchinson’s wonderful TtH TeX-to-HTML converter) and then pops up the HTML with the exercise in a browser. MathML support is needed for this version of the HTML exercise, e.g., by using the Firefox browser.

How it works

The tstat.Rnw is a relatively simple Sweave file: It first draws some random data and then uses the textbook formula to compute the correct t statistic. Subsequently, there are two LaTeX environments with the {question} and {solution}, respectively. Finally, some simple metainformation is provided in LaTeX style (but commented):

<>=
## DATA GENERATION
n n}}}
    = \frac{\Sexpr{Mean} - \Sexpr{mu}}{\sqrt{\frac{\Sexpr{Var}}{\Sexpr{n}}}}
    = \Sexpr{tstat}.
  \end{eqnarray*}
  The absolute value of the $t$~test statistic is thus equal to
  $\Sexpr{format(abs(tstat), nsmall = 3)}$.
\end{solution}

%% META-INFORMATION
%% \extype{num}
%% \exsolution{\Sexpr{format(abs(tstat), nsmall = 3)}}
%% \exname{t statistic}
%% \extol{0.01}

After running Sweave() on it, the resulting LaTeX code has the random numbers filled in and the data-generating code omitted:

\begin{question}
  A machine fills milk into $500$ml packages. It is suspected that the 
  machine is not working correctly and that the amount of milk filled differs 
  from the setpoint $\mu_0 = 500$. A sample of $226$ packages 
  filled by the machine are collected. The sample mean $\bar{y}$ is equal to 
  $517.2$ and the sample variance $s^2_{n-1}$ is equal to $262.56$.

  Test the hypothesis that the amount filled corresponds on average to the 
  setpoint. What is the absolute value of the $t$~test statistic?
\end{question}

\begin{solution}
  The $t$~test statistic is calculated by:
  \begin{eqnarray*}
    t & = & \frac{\bar y - \mu_0}{\sqrt{\frac{s^2_{n-1}}{n}}}
    = \frac{517.2 - 500}{\sqrt{\frac{262.56}{226}}}
    = 15.958.
  \end{eqnarray*}
  The absolute value of the $t$~test statistic is thus equal to
  $15.958$.
\end{solution}

%% META-INFORMATION
%% \extype{num}
%% \exsolution{15.958}
%% \exname{t statistic}
%% \extol{0.01}

The LaTeX code is then read into R, can be transformed into HTML using tth(), embedded into a simple HTML file, and then shown in a browser:

Support for learning management systems

The same HTML code shown in the example above can also be embedded into learning management systems. These typically have XML-based exchange formats, e.g., Moodle uses its own Moodle XML while OLAT and several other systems use the international IMS QTI 1.2 standard. The package provides functions exams2moodle() and exams2qti12() which provide output files that can be easily imported into Moodle and OLAT, respectively. As a result, the tstat.Rnw looks in Moodle like this: Note that this now has a field for entering the answer (and only after that the correct solution can optionally be shown). To generate a Moodle quiz with three random replications for five exercises drawn from collections of different templates, consider the following example (based on templates provided within the package):

myexam <- list(
  "boxplots",
  c("confint", "ttest", "tstat"),
  c("anova", "regression"),
  "scatterplot",
  "relfreq")
exams2moodle(myexam, n = 3)

This produces a file moodlequiz.xml that can be imported into the question bank in Moodle. (In this step, the HTML markup for formulas and graphics is not formatted by Moodle but don't worry: the final quiz will render correctly.) Subsequently, all replications of each exercise can be added as "random" questions into the quiz. Some more configuration of the quiz as a whole can then be done in Moodle but otherwise everything else is ready for the participants of the quiz.

If you want to use this for your own course...

• You need an installation of a suitable learning management system but many universities have support for this anyway.
• You need to build your collection of Sweave exercise templates. This is, of course, the most work. However, the package provides a suite of examples that can be helpful as starting points.
• You need to export the exams/quizzes/assessments from R to the correct format for the learning management system. Moodle and OLAT are directly supported. But through QTI 1.2 other systems could, in principle, be interfaced as well. We haven't tested this yet, though.

More details are provided in the working paper and we also provide a support forum on R-Forge if you experience any problems etc.