This blog post explains how to test a Shiny app using shinytest and testthat packages. Basic knowledge about Shiny apps and the principle of unit testing using testthat is useful, but not required here.

Example of a Shiny app

The packages shiny (current version: 1.1.0), testthat (2.0.0) and shinytest (1.3.0) are required for the test presented here and may be installed with install.packages().

Below is a minimal example of a Shiny app (app.R) to be be tested. The app has only a single numerical input and a text output. The entered number n is squared and the result is shown as text.

library(shiny)

ui <- fluidRow(title = 'Minimal app',
               numericInput("num_input", "Please insert a number n:", 0),
               textOutput('text_out')
               )

server <- function(input, output, session) {
  result <- reactive(input$num_input^2)
  output$text_out <- renderText(
    paste("The square of the number n is: n² =", result())
    )
}

shinyApp(ui, server)

And this is how the app looks like:

What is shinytest?

The package shinytest provides automatic testing of a Shiny app. Both the “appearance” of the app, as well as its “internal” state during the program flow can be examined. An interactive user interface can be used to create snapshots (more precisely, reference snapshots) as well as a test file. The test file contains the code required for later generation of the snapshots. Each test run creates new snapshots and compares them to the reference snapshots to automatically detect unexpected behavior of the Shiny app. More about the normal workflow with shinytest can be found here. This blog post, however, describes a different approach of testing; Namely testing with shinytest and testtthat combined [*].

[*] Another way of testing uses the function expect_pass (see ?expect_pass), which needs a test file created by shinytest (as well as reference snapshoots) as an input argument. While this allows quick testing, the approach presented here enables more detailed and specific tests.

Testing Shiny apps using shinytest & testthat

shinytest has the class ShinyDriver (see ?ShinyDriver) which opens the Shiny app in a new R-Session as well as an instance of PhantomJS, and connects the two. PhantomJS is a headless web browser that can be operated by JavaScript. The ShinyDriver object is equipped with various methods that enable, among other things, setting/getting values of different variables (inputs or outputs) in the Shiny app. That way, we can assign arbitrary values to the input variables, “manually” (without the usual user interface of the Shiny app), and then get the output variables.

Example of a test

In the following test, the variable num_input is set to 30 and consequently the variable text_out is tested to see if it becomes the string “The square of the number n is: n² = 900”. More on testing with testthat can be found here.

library(shinytest)
library(testthat)

context("Test Shiny app")

# open Shiny app and PhantomJS
app <- ShinyDriver$new("<path to app.R>")

test_that("output is correct", {
  # set num_input to 30
  app$setInputs(num_input = 30)
  # get text_out
  output <- app$getValue(name = "text_out")
  # test
  expect_equal(output, "The square of the number n is: n² = 900")  
})

# stop the Shiny app
app$stop()

Using the expectation functions of the package testthat it is thus easily possible to test the functionalities of the Shiny app. An advantage of this is that when calling devtools::test() both the tests of the Shiny app and other unit tests are taken into account.

Deeper insights – Exported variables and HTML widgets

Within the server function, we can also define new variables (in addition to the usual inputs and outputs) and export them to be “visible” for shinytest and allow for more detailed examination of the app’s workflows. As an example for the Shiny app shown above, we can save a list of all the numbers n entered and export them as a variable inputs_list (please see the code below).

For different HTML widgets the method findElement can be used via app$findElement(xpath ="here the XPATH") using the XPath parameter. For example, if notifications are used with the showNotification() function in the Shiny app, they can be identified with xpath = "//*[@id=\"shiny-notification-panel\"]" and can be tested correspondingly.

Here is how to export a variable in the Shiny app and display notifications using showNotification().

library(shiny)

# same ui as above
ui <- fluidRow(title = 'Minimal app',
               numericInput("num_input", "Please insert a number n:", 0),
               textOutput('text_out')
               )

server <- function(input, output, session) {
  result <- reactive(input$num_input ^ 2)
  output$text_out <- renderText(
    paste("The square of the number n is: n² =", result())
    )
  # initialising the exported list
  inputs_list <- c()
  observeEvent(input$num_input, {
    # new input will be added to inputs_list
    inputs_list <<- c(inputs_list, input$num_input)
    # show notification
    showNotification(HTML(result()), duration = NULL)
    })
  # export inputs_list
  exportTestValues(inputs_list = {inputs_list})
}

shinyApp(ui, server)

For example, a test might look like this:

library(shinytest)
library(testthat)

context("Test Shiny app")

# open Shiny app and PhantomJS
app <- ShinyDriver$new("<path to app.R>")

test_that("inputs_list is exported correctly", {
  # multiple inputs
  app$setInputs(num_input = 1)
  app$setInputs(num_input = 7)
  app$setInputs(num_input = 42)

  # get exported variable inputs_list
  exported_list <- app$getAllValues()$export$inputs_list

  # test (0 was the initial value)
  expect_equal(exported_list, c(0, 1, 7, 42))  
})

test_that("Notifications include correct text", {
  # identify HTML widget with XPath
  popup <- app$findElement(xpath = "//*[@id=\"shiny-notification-panel\"]")

  # test notification text
  testthat::expect_equal(popup$getText(), "×\n0\n×\n1\n×\n49\n×\n1764")
})

# stop the Shiny app
app$stop()