After posts exploring READMEs, URLs in DESCRIPTION, today we shall look at another important aspect of documentation: examples in the manual pages of your functions! Why write them, how to write them, and how to control their execution in different contexts (CRAN or not ????).

Why write examples in manual pages?

We’ve made the point for users discovering packages through READMEs. Now, for functions, they might type ?foo or open a pkgdown reference page and go straight to the example parts rather than read the previous sections (about parameters, details, etc).

Users might do that either to get a quick feel for how the functions work, or refresh their memories of it. Say you discovered the great pkgsearch package and decided to try it out. You might look at the local or online reference index and then nagivate to the help for package_search() to read examples.

Ideally your package should have examples for every function. The long-form docs (README, vignette) can still show functions working together and give a bigger picture, but adding executable code to your manual pages is crucial.

How to run examples

As the user of an R package, how can you run examples you see in a package doc? (This is useful to know as a developer too!)

How to run all examples from a man page

Locally, you could run

example("pkg_search", package = "pkgsearch")

or

library("pkgsearch")
example("pkg_search")

As a side-effect, this makes all objects created in the example(s) available in your workspace.

How to run specific examples from a man page

If you want to run only one of the examples you’re reading

• You can select the lines and then run Command+Enter (Ctrl+Enter on Linux), be it in the R GUI or in RStudio. Below is a screenshot created by Mine Çetinkaya-Rundel when she curated the We are R-Ladies Twitter account.

????So long copy paste,
????Hello Command + Enter!

Did you know that you can run code from examples in help files with Command + Enter? Works in the #rstats GUI and in @rstudio. pic.twitter.com/i704xzr91p

— We are R-Ladies (@WeAreRLadies) November 20, 2018

• From an online pkgdown website, you’d use the copy button and then paste the result in your R console.

How to write examples?

In an Rd file, examples are inside \examples{} e.g.

\examples{
  # basic usage of foo
  foo(basic = TRUE)
  # elaborate usage of foo
  foo(basic = FALSE)
}

But usually nowadays docs are generated by roxygen2 so your examples can either

• exist after the @examples tag

#' @examples
#' # basic usage of foo
#' foo(basic = TRUE)
#' # elaborate usage of foo
#' foo(basic = FALSE)

• exist in an R script that you’d source from the docs i.e.

#' @example man/examples/foo.R

and in man/examples/foo.R

# basic usage of foo
foo(basic = TRUE)
# elaborate usage of foo
foo(basic = FALSE)

The former solution (@examples tag: examples with other docs for the function inside the function R file) means you can work on the function code and examples at the same time more smoothly, but all your lines start with #'; the latter solution (@example tag without an s: examples in dedicated files) means you can write examples as usual R code without #'.

How to know there are enough examples?

You could judge whether the man page of each function shows the most important behaviours by reading said man pages. You could also use the output of covr::package_coverage(<path-to-your-package>, type = "examples") to reveal major gaps!

How to safeguard examples?

Examples are supposed to be executable i.e. to run without error. As writing in “Writing R Extensions”, “By default, text inside \examples{} will be displayed in the output of the help page and run by example() and by R CMD check.”. R CMD check’s running example could be described as a smoke test.

Now comes one of the trickiest aspects of creating examples for the manual pages: sometimes the two goals of examples (examples for the user to try or read; examples for smoke testing) can conflict.

• Examples can show an error, to exemplify the behaviour of the function in such cases.
• Examples may need user setup such as having an API key available.
• Examples can be too long for CRAN.

Thankfully there are few toggles that you can use for your examples. From “Writing R extensions”, “You can use \dontrun{} for text that should only be shown, but not run, and \dontshow{} for extra commands for testing that should not be shown to users” and “Finally, there is \donttest, used (at the beginning of a separate line) to mark code that should be run by example() but not by R CMD check (by default: the option --run-donttest can be used).”. In the help files the user reads, \dontshow{} lines are well, not shown; dontrun{} blocks come between ## Not run: and ## End(Not run) lines; \donttest{} lines look like all other lines. (See lookup::lookup(tools::Rd2HTML)) That’s a lot of information to process, but roxygen2 docs feature a handy reference table.

To complicate things a bit more, example() and R CMD check have toggles too! You could run example(<topic>, run.dontrun = TRUE) and R CMD check with the --run-donttest options. ????

Here’s a summary

• Example that works out of the box but long or interactive ➡️ donttest (used 2,019 times in CRAN packages at the time of writing).

• Example that doesn’t work out of the box or doesn’t work at all ➡️ dontrun (used 19,065 times in CRAN packages at the time of writing).

For a CRAN submission you might be told to use donttest for examples using a token.

• Example where you want to sneakily subset of the example data (to make it run faster) ➡️ dontshow (used 1,209 times in CRAN packages at the time of writing). Note that in that case the user gets different results when copy-pasting the example vs. running example().

Depending on why you choose to pack some examples in \dontrun, you might enjoy the run_dont_run option of pkgdown::build_site(), e.g. if your examples depend on having authentication tokens that are present when building the pkgdown website.

How to check your examples?

You could run R CMD check with the options you want. Or you could use

• devtools::run_examples() that has run and test toggles.

• testthat::test_examples(). This function doesn’t pass any argument to tools::Rd2ex() that it uses to extract the examples from the docs, so it will test donttest examples but not test dontrun examples (commentDontrun = TRUE, commentDonttest = FALSE). You can use test_examples() in a testthat test file to have examples run by devtools::test() but note that the docs state that “Generally, this is redundant with R CMD check, and is not recommended in routine practice.”.

What can’t example code do?

Two items from the CRAN policies should be borne in mind when creating examples (and when choosing dontrun/donttest/dontshow toggles ????).

• “Examples should run for no more than a few seconds each: they are intended to exemplify to the would-be user how to use the functions in the package.” ⌛️

• “The code and examples provided in a package should never do anything which might be regarded as malicious or anti-social.” ???? The policies give examples of such behaviour such as “Packages should not write in the user’s home filespace (including clipboards), nor anywhere else on the file system apart from the R session’s temporary directory”. So make good use of tempdir()!

Conclusion

In this post we answered questions around the preparation of examples for manual pages of R packages. If you wish there were even more flexibility, be on the lookout for future developments for examples written with roxygen2, for instance check out the WIP roxygen2labs package.