Automating End-to-End Cypress Tests in Rhino: A Guide to Seamless UI Testing for Shiny Apps

No one wants to break an application! And part of the process of developing a quality R/Shiny dashboard is making sure that new features or bug fixes do not create new problems.

This can be done by testing the application, something that developers perform manually for the task at hand, but overtime this process becomes too time consuming and inconsistent when looking at all the components of an application.

Here is where automated tests enter the picture. These can range from small-scall unit tests that verify expected output from function calls or include interface tests that simulate user interaction with the application. 

End-to-End UI Testing a Rhino Shiny app

In this article, we will focus on user interface (UI) testing and how we can take advantage of the Rhino framework to seamlessly create end-to-end tests (e2e) for an R/Shiny application using Cypress.

Rhino provides the infrastructure to create enterprise ready applications and includes ready to use end-to-end tests via the Cypress testing framework. Cypress is used to test modern applications in the browser and is not specific to Shiny. For Shiny-specific testing, consider shinytest2.

This article will show an example of how to work with Cypress. We can start by setting up the environment by installing R and Node.js (needed for Cypress). This tutorial uses RStudio IDE for demonstration purposes, but it is an optional requirement.

TOC:

• R and Node.JS
• Setting Up Rhino
• Make It Look Blue: CSS/SASS on Shiny
• Preparing for the First Cypress Test
• Writing the First Cypress Test
• Building A Shiny Dashboard with New Features
• Writing Tests for New Shiny App Features
• Writing a Message Test
• Writing a Click Test
• Writing a Map Interaction Test
• What Happens When the App Changes?
• Continuous Integration
• Writing a Map Interaction Test

R and Node.JS

R can be installed from the r-project.org following the instructions on their download page.

Node.JS is needed to run Cypress tests and the instructions for the different operating systems can be found here (Windows, MacOS, Linux and Docker). 

To test if Node.JS is correctly installed we can call the command below on the R / RStudio console to check the installed version:

> system(“node -v”)

Tip for Linux Users: When using the node version manager (NVM) you should be aware of a known issue with RStudio and how to mitigate it.

Setting Up Rhino

Once the environment is ready, change the working directory to an empty folder or start a new project on RStudio, and initialize Rhino.

> install.packages(“rhino”)
> rhino::init()

The boilerplate code will add a few files and directories to the working directory:

• renv environment to manage installed packages and versions
• Github actions for continuous integration (including for e2e tests)
• Shiny folder structure on app/ directory
• Test folder for e2e and unit tests

The boilerplate Shiny application is ready to be run and will show a default dashboard. We will replace main.R with a minimal example that renders “Hello!” via the `renderText()` Shiny function. We will do this by replacing the `app/main.R` with the code below:

# app/main.R

box::use(
  shiny[h3, textOutput, renderText, fluidPage, fluidRow, moduleServer, NS],
)

#' @export
ui <- function(id) {
  ns <- NS(id)
  fluidPage(fluidRow(h3(class = "title", textOutput(ns("header")))))
}

#' @export
server <- function(id) {
  moduleServer(id, function(input, output, session) {
    output$header <- renderText("Hello!")
  })
}

When running the Shiny application with the default Shiny command we will be greeted:

> shiny::runApp()

Boilerplate Rhino dashboard

Make It Look Blue: CSS/SASS on Shiny

Let’s make the application look better than the default blank style by adding a snippet of CSS/SASS on `app/styles/main.scss` and build the CSS file. Besides an integration with Cypress e2e tests, Rhino also boasts a seamless integration with SASS.

body {
  margin: 0;
  background-color: #15354a;
  padding: 1em;

  .container-fluid {
    border-radius: 0.5em;
    max-width: 1200px;
    background-color: white;

    .row {
      padding: 1em;
    }
  }
}

Build the CSS for the dashboard using the command below:

> rhino::buil_sass()
> shiny::runApp()

# The browser page might need to be reloaded to refresh the cache

Boilerplate Rhino dashboard with simple styling

Preparing for the First Cypress Test

As we can see from the previous screenshot, the R/Shiny dashboard only shows a simple ‘hello’ message. We can start by testing this via Cypress to understand how to create simple tests.

The tests in Cypress are written in JavaScript and are based on the “cy” object. It is very intuitive to replicate a user’s manual action. It mostly revolves around looking for elements, checking their content and properties, whilst performing actions.

For the first test, we want to check if it contains the title showing “Hello!” text. We can modify the `app.spec.js` file located at `tests/cypress/integration` and prepare by creating an empty test with a descriptive title.

describe('app', () => {
  # ...

  it('Hello text appears', () => { })
})

Let’s make sure that in the beginning, Cypress will open the dashboard. To achieve that, add a `beforeEach()` statement at the top of the test so that before each test (each “it”) Cypress will go to the application root URL.

describe('app', () => {
  beforeEach(() => {
    cy.visit('/')
  })
  it('Hello text appears', () => { })
})

Tip: You don’t need to provide the full address, as the base URL is already pre-configured by Rhino (see the `tests/cypress.json` configuration file)

Writing the First Cypress Test

The test is going to use two Cypress commands: `get()` and `should()`. The first one will look for an element using a given CSS selector and the second will check if the selected element has the expected property.

First, we need to use a CSS selector that will point Cypress to the correct button. This can be done using the browser’s ‘Developer Tools’, that can be called ‘Inspector’, ‘Explorer’, or ‘Elements’ (depending on the browser). We can learn more about ‘Developer Tools’ available in the browser from this article by Mozilla.

The following steps can be generally performed to find a CSS selector:

• Run the application `shiny::runApp()`
• Open it in the browser
• Right-click on the “Hello!” text, and select Inspect

We should be able to see the HTML structure of the webpage with the text highlighted. The selected element is an HTML H3 tag with a DIV tag inside. We can use any CSS selector that allows to select the element, such as the class of the H3 or the id of the DIV.

Let’s pass two ways to select the text to Cypress’ `get()` function (using the H3 tag with the “title” class as well as the class id of the Shiny element).

describe('app', () => {
  beforeEach(() => {
    cy.visit('/')
  })

  it('starts', () => {})

  it('hello text appears', () => {
    cy.get("h3.title").should("contain", "Hello")
    cy.get("#app-header").should("contain", "Hello")
  })
})

This example shows two different methods of looking for the text, with the second being more specific by searching the element for the exact id.

> rhino::test_e2e()

Tip: If you interrupt the `rhino::test_e2e()` the first time it runs and it shows errors the second time, then delete the hidden folder named `.rhino` on the root directory.

Tip: Running the tests with `interactive = TRUE` allows you to inspect the tests using the Cypress interface (see the first GIF on the article

Building A Shiny Dashboard with New Features

We are building a simple dashboard with 3 main features (and later create additional tests):

• Message that only shows after clicking a button
• Counter that is increased every time a button is clicked
• Map that adds random location markers with each counter click

Let’s take advantage of modules in Shiny and create one for each of the features in the `app/view/` directory (`clicks.R`, `map.R` and `message.R`). We also need to update `main.R` to include the new modules.

# app/view/message.R
box::use(
  shiny[actionButton, div, moduleServer, NS, renderText, req, textOutput],
)

#' @export
ui <- function(id) {
  ns <- NS(id)

  div(
    class = "message",
    actionButton(
      ns("show_message"),
      "Show message"
    ),
    textOutput(ns("message_text"))
  )
}

#' @export
server <- function(id) {
  moduleServer(id, function(input, output, session) {
    output$message_text <- renderText({
      req(input$show_message)
      "This is a message"
	})
  })
}


# app/view/clicks.R
box::use(
  shiny[actionButton, div, moduleServer, NS, renderText, textOutput, reactive],
)

#' @export
ui <- function(id) {
  ns <- NS(id)
  div(
    class = "clicks",
    actionButton(ns("click"), "Click me!"),
    textOutput(ns("counter"))
  )
}

#' @export
server <- function(id) {
  moduleServer(id, function(input, output, session) {
    output$counter <- renderText(input$click)
    return(reactive(input$click))
  })
}


# app/view/map.R
box::use(
  shiny[div, moduleServer, NS, reactive, observe, bindEvent],
  leaflet[
	leafletOutput, renderLeaflet, leaflet, addProviderTiles, providers,
	providerTileOptions, addMarkers, leafletProxy, setView, fitBounds
  ],
  stats[rnorm],
)

#' @export
ui <- function(id) {
  ns <- NS(id)
  div(class = "maps", leafletOutput(ns("map_random")))
}

#' @export
server <- function(id, input_click) {
  moduleServer(id, function(input, output, session) {
    output$map_random <- renderLeaflet({ leaflet() |>
        fitBounds(lng1 = -11.16, lat1 = 34.9, lng2 = 22.4, lat2 = 58) |>
        addProviderTiles(
          providers$Stamen.TonerLite,
          options = providerTileOptions(noWrap = TRUE)
        )
    })

    observe({
      points <- cbind(rnorm(1, mean = 10.6, sd = 10), rnorm(1, mean = 49.1, sd = 3)) leafletProxy("map_random") |> addMarkers(data = points)
    }) |>
      bindEvent(input_click())
  })
}


# app/main.R
box::use(
  shiny[h3, textOutput, renderText, column, fluidPage, fluidRow, moduleServer, NS],
)

box::use(
  app/view/clicks,
  app/view/message,
  app/view/map,
)

#' @export
ui <- function(id) {
  ns <- NS(id)
  fluidPage(
	fluidRow(h3(class = "title", textOutput(ns("header")))),
	fluidRow(
  	column(width = 6, clicks$ui(ns("clicks"))),
  	column(width = 6, message$ui(ns("message")))
	),
	fluidRow(map$ui(ns("map")))
  )
}

#' @export
server <- function(id) {
  moduleServer(id, function(input, output, session) {
    output$header <- renderText("Hello!")
    input_click <- clicks$server("clicks")
    message$server("message")
    map$server("map", input_click)
  })
}

Install Leaflet (requirement for map module)

We need to install the “leaflet” library before running the application. This can be done via renv:

> renv::install("leaflet")

The library should be added to the `dependencies.R` file in the root directory.

# This file allows packrat (used by rsconnect during deployment) to pick up dependencies.
library(rhino)
library(leaflet)

Running the Shiny Application

We can check how the new modules look by running the dashboard via:

> shiny::runApp()

Writing Tests for New Shiny App Features

We are going to test the new features by checking:

• The initial message is not visible (unless clicked)
• The buttons exist and are visible (“Click me!” and “Show message”)
• Counter starts at 0
• Every time the button on the left is clicked it will:
  • Increase the counter
  • Add new marker to map

We can start with `message` tests by creating a new file for the specification `message.spec.js`.

Writing a Message Test

The first thing to do is to create a test file: `tests/cypress/integration/message.spec.js` with a similar structure as the simple example above. It should include the boilerplate for the tests (the “it”) and indicate to Cypress that it should visit the root page before running each test.

# tests/cypress/integration/message.spec.js
describe("Show message", () => {
  beforeEach(() => {
	cy.visit('/')
  })

  it("'Show message' button exists", () => { });

  it("'Show message' button shows the message", () => { });
});

The “‘Show message' button exists” workflow should start by finding the button using a CSS selector and then check:

• Button is visible
• Button has the expected content

We will use the `should(“be.visible”)` and `should(“have.text”, “<some text>”)` to perform these checks.

The second test “‘Show message’ button shows the message” will need to click the button using  Cypress’ `click()` method and lookup if the message appears. We will also check that the message is not visible before clicking.

# tests/cypress/integration/message.spec.js
describe("Show message", () => {
  beforeEach(() => {
	cy.visit('/')
  })

  it("'Show message' button exists", () => {
    // check if button exists and has correct label
    cy.get(".message button")
      .should("be.visible")
      .should("have.text", "Show message");
  });

  it("'Show message' button shows the message", () => {
    // ensure that the message is not visible without clicking the button
    cy.get("#app-message-message_text").should("not.be.visible");

      // click on message button to display message
    cy.get(".message button").click();
    cy.get("#app-message-message_text")
      .should("be.visible")
      .should("have.text", "This is a message");
  });
});

Message from Cypress after running the message specification tests

Writing a Click Test

Next we will create the specification for the “click” counter functionality, by creating the file `tests/cypress/integration/click.spec.js`.

This workflow will look at 3 parts of the interface:

1. The button is visible and has the expected content
2. The initial value for the counter is 0 (zero)
3. When clicking the button it should verify that the counter on the dashboard increases
   • We will test with an arbitrary number of clicks

We will skip the boilerplate code with 3 empty tests and share the code. Note that there is a click on the message button, just to make sure that clicks on other buttons don’t affect the counter.

# tests/cypress/integration/clicks.spec.js
describe("Counting clicks", () => {
  beforeEach(() => {
	cy.visit("/");
  });

  it("Has a 'Click me!' button", () => {
    cy.get(".clicks button")
      .should("have.text", "Click me!")
      .should("be.visible");
  });

  it("Counter starts at zero", () => {
    cy.get("#app-clicks-counter")
      .should("have.text", "0")
      .should("be.visible");
  })

  it("Counter increases with clicks", () => {
    cy.get(".clicks button").as("button");

    for (let i = 0; i < 10; i++)
      cy.get("@button").click();

    cy.get(".message button").click();

    cy.get("#app-clicks-counter").should("have.text", "10");
  });
});

When running the tests in interactive mode, we can observe the test running automatically and we can also inspect each step.

Writing a Map Interaction Test

The map specification will be similar to the click, but instead of looking for the counter content it should check the number of markers on the map.

We can start by adding a few markers on the map and inspecting one of them to find a valid CSS selector. We can use the “leaflet-marker-icon” class and count the number of markers.

We will create a specification in `tests/cypress/integration/map.spec.js` with a test that clicks on the button creating a bunch of markers and then counts the number of markers that exist on the map. As with the other specification, we need to visit the root page before each test.

# tests/cypress/integration/map.spec.js
describe("Counting map markers", () => {
  beforeEach(() => {
    cy.visit("/");
  });

  it("Has no markers", () => {
    cy.get("#app-map-map_random .leaflet-marker-icon").should('have.length', 0)
  });

  it("Each click adds a marker", () => {
    cy.get(".clicks button").as("button");

    for (let i = 0; i < 100; i++)
      cy.get("@button").click();

    // Clicking on show message shouldn't add or remove any markers
    cy.get("#app-message-show_message").click()

    cy.get("#app-map-map_random .leaflet-marker-icon")
      .should('have.length', 100);
  });
});

The tests use the “have.length” property of the “should()” method to check the number of markers on the map. We also check if there are no markers before clicking on the counter.

Message from Cypress after running the map test specifications. Note that it takes around 8 seconds to finish the test, as the interface needs to click on the button 100 times. And then it will wait for the markers to have the correct length for a default timeout.

How Long Does Cypress Wait for Elements?

Cypress will wait for a default of 4000 ms between commands and this timeout can be adjusted globally for a specific test or to a single command.

The example below shows 3 different methods to change the timeout to 10 seconds:

// change the overall configuration
Cypress.config('defaultCommandTimeout', 10000);

// Change for a single test
it('Should do something', { defaultCommandTimeout: 10000 }, () => {
  // ...
})

// Change for a single command
cy.get('ELEMENT', { timeout: 10000 })

What Happens When the App Changes?

Let’s now think about a scenario in which, during the development of the application, a new feature has been introduced. This feature works fine; however, it has unfortunately introduced some unexpected changes to other features.

We will simulate this by a small modification in the `app/view/clicks.R`and add 1 to the counter:

# ...
server <- function(id) {
  moduleServer(id, function(input, output, session) {
    output$counter <- renderText(input$click + 1)
    return(reactive(input$click))
  })
}

If you now run tests, you will see that the two tests fail, the one that verifies the initial counter value, and the test that counts the number of clicks.

Message from Cypress after running the unsuccessful tests comparing the expected result with the actual result.

This way we can spot the (unexpected) change in the application behavior in an automated way. Without end-to-end tests, this could go unnoticed, since the app still works. There are no errors, it is just the logic that has been changed in a way we didn’t want it to be.

Continuous Integration

Here’s a question for you: Do we need to run those tests each time there’s a new feature or fix?

Answer: Yes, you should.

Fortunately, this can be automated! Rhino comes with a setup for GitHub Actions which will run those tests and inform you if there is a problem. 

Tip: You can even block merging changes that don’t pass those automated checks.

What do we need to do to get all of this? When using GitHub.. nothing! The Rhino application comes with a GitHub Actions configuration `.github/workflows/rhino-test.yml` that includes end-to-end tests!

Screenshot of the Github Action running

Cypress Testing in Rhino-Built Shiny Apps

Adding end-to-end tests to your application can ensure the quality and help catch bugs early in the process of development. Ultimately, this can save you time and resources (not to mention headaches!). Rhino comes with a powerful setup that utilizes Cypress to check the behavior of your application.

But what if you want to use another solution: shinytest2?

You can do that! shinytest2 works out of the box in Rhino applications! You can learn more on the Rhino tutorials page.

And feel free to explore our blog to learn about the differences between Cypress and shinytest2.

The post appeared first on appsilon.com/blog/.