Building Big Shiny Apps — A Workflow 2/2

< g class="gr_ gr_5 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar only-ins doubleReplace replaceWithoutSep" id="5" data-gr-id="5">Second part of the blog transcription of the talk I’ve given during the < g class="gr_ gr_3 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="3" data-gr-id="3">eposter session of the < g class="gr_ gr_4 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="4" data-gr-id="4">rstudio:< g class="gr_ gr_6 gr-alert gr_gramm gr_inline_cards gr_run_anim Style replaceWithoutSep" id="6" data-gr-id="6">:conf(2019).

Read the first part here.

Building Big Shiny Apps: step by step

Step 1: Designing

Don’t rush into coding. I know you want < g class="gr_ gr_10 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-del replaceWithoutSep" id="10" data-gr-id="10">to, because it’s what we like to do and what we are good at. But before entering the coding marathon, take time to think about the application and the way it will be deployed and used. Take a pen and a piece of paper and draw the app. Talk about it with the people who will use the app, just to decipher what they actually need. Take a moment to talk with < g class="gr_ gr_9 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar only-del replaceWithoutSep" id="9" data-gr-id="9">the IT. Here are some questions you can ask:

• “Who are the end users of the app?” — This will help you know if the end users are tech literate or not, and what they aim to achieve with the app.
• “How frequently will they use the app?” — The small details of the design & the UI of an app you use on a daily basis < g class="gr_ gr_134 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Grammar multiReplace" id="134" data-gr-id="134">is more crucial than when the app is used once a month.
• “What level of complexity and personalization do the users really need?” — People writing app specifications sometimes want more functionalities than what is actually needed by the users.
• “What level of interactivity do you want, and to what extent is it central?” — People love interactive graphs and like when things automatically sync with each other. Yet these two can make the app slower, without any significant gain. For example, being reactive to a selectInput() or a sliderInput() can lead to too much computation: maybe the user will not succeed to choose the right input the first, second or third time… So let them do their choice, and add a button so that they can validate when they are ready.
• “How important is it that the app is fast?” — Should you spend a lot of time optimizing the little things?
• etc.

Asking questions, taking notes, and drawing the app help you have a good idea of what is expected and what you have to do now.

So, next step!

Step 2: Prototype

I like to go “UI first”. For two main reasons:

• Once the UI is set, there is no “surprise implementation”. Once we agree on what elements there are in the app, there is no sudden “oh the app needs to do that now”.
• A pre-defined UI allows every person involved in the coding to know which part of the app they are working on. In other words, when you start working on the backend, it’s way much easier to work on a piece you can visually identify and integrate < g class="gr_ gr_9 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar multiReplace" id="9" data-gr-id="9">in a complete app scenario.

So yes, spend time writing a front-end prototype in lorem ipsum. And good news, we’ve got a tool for you: it’s < g class="gr_ gr_7 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="7" data-gr-id="7">called {shinipsum}< g class="gr_ gr_7 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="7" data-gr-id="7">. The main goal of this package is to create random Shiny elements that can be used to draw a UI, without actually do any heavy lifting in the backend.

Hence, once you’ve got a draft of your app on a piece of paper, you can then move to the “< g class="gr_ gr_3 gr-alert gr_spell gr_inline_cards gr_disable_anim_appear ContextualSpelling ins-del multiReplace" id="3" data-gr-id="3">ipsum-UI” stage: building the front-end of the app, and filling it with random Shiny elements, with functions < g class="gr_ gr_16 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="16" data-gr-id="16">like random_ggplot()< g class="gr_ gr_16 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="16" data-gr-id="16"> or random_DT().

Another package that can be used to do that is {fakir}}. This package is designed to create fake data frames, primarily for teaching purpose, but it can be used for inserting data in a shiny prototype.

Step 3: Build

Now the UI and the features are set, time to work on the backend

This part is pretty standard — everybody can now work on the implementation of the functions that process the app inputs, in their own modules. As the UI, functionalities < g class="gr_ gr_44 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-ins replaceWithoutSep" id="44" data-gr-id="44">and modules have been defined in the previous steps, everyone (well, in theory) knows what they have to work on.

And also, as said before, there should be no “surprise implementation”, as the app has been well defined before.

Step 4: Secure

Securing your app means two things: testing, and locking the application environment.

So first, be sure to include tests all along the building process — just like any other R code. As the app is contained in a package, you can use standard testing tools for testing the business logic of your app — as said in the first part, it’s important to split the backend functions and algorithm from the user interface. That means that these backend functions can run outside of the application. And yes, if they can run outside of the app, they can be tested the standard way, < g class="gr_ gr_9 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="9" data-gr-id="9">using {testthat}< g class="gr_ gr_9 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="9" data-gr-id="9">.

When it comes to testing the front end, you can try < g class="gr_ gr_5 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="5" data-gr-id="5">the {shinytest}< g class="gr_ gr_5 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="5" data-gr-id="5"> package from RStudio, if you need to be sure there is no visual regression all along the project development. {shinyloadtest}, on the other hand, tests how an application behaves when one, two, three, twenty, one hundred users connect to the < g class="gr_ gr_4 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-del replaceWithoutSep" id="4" data-gr-id="4">app, and gives you a visual report about the connections and response time of each session.

One other tool I like to use is Katalon Studio. It’s not R < g class="gr_ gr_39 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-del replaceWithoutSep" id="39" data-gr-id="39">related, and can be used with any kind of web app. How it works is quite simple: it opens your browser where the Shiny app runs, and record everything that happens. Once you stop the recording, you can relaunch the app and it will replay all the events it has recorded. And of course, you can specify your own scenario, define your own events, etc. It’s not that straightforward to use, but once you get a good grasp of how it works, it’s a very powerful tool.

Secondly, secure your app means that it can be deployed again any time in the future — in other words, you have to ensure you’ve got a proper handle on the required R version, and of the packages versions which are required to run your app. That means that you have to be aware that upgrading a package might break your app — so, provide an environment that can prevent your app from breaking when a package gets updated. For that, there is < g class="gr_ gr_20 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation replaceWithoutSep" id="20" data-gr-id="20">of course Docker, but also R specific tools < g class="gr_ gr_7 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="7" data-gr-id="7">like {packrat}< g class="gr_ gr_7 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="7" data-gr-id="7">, custom CRAN repositories or package manager.

Step 5: Deploy

Tools for deployment are not the subject of this blog post so I won’t talk about this in details (remember, we are talking about building ), but our two tools of choice are Docker & ShinyProxy, and RStudio Connect.

Building Big Shiny Apps: an introduction to {golem}

Ok, that’s a lot of things to process. Is there a tool that can help us simplify this workflow? Of < g class="gr_ gr_5 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-ins replaceWithoutSep" id="5" data-gr-id="5">course there is, and it’s < g class="gr_ gr_6 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="6" data-gr-id="6">called {golem}< g class="gr_ gr_6 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="6" data-gr-id="6">.

It can be found at https://github.com/ThinkR-open/golem

{golem} is an R package that implements an opinionated framework for building production-ready Shiny apps. It all starts with an RStudio project, which contains a predefined setup for building your app. The idea is that < g class="gr_ gr_550 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="550" data-gr-id="550">with {golem}< g class="gr_ gr_550 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="550" data-gr-id="550">, you don’t have to focus on the foundation of your app, and can spend your time thinking about what you want to do, not about how to do it. It’s built on top of the working process we’ve developed at < g class="gr_ gr_498 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Punctuation only-del replaceWithoutSep" id="498" data-gr-id="498">< g class="gr_ gr_217 gr-alert gr_spell gr_inline_cards gr_disable_anim_appear ContextualSpelling ins-del multiReplace" id="217" data-gr-id="217">ThinkR, and tries to gather in one place the functions and tools we’ve created for building applications designed for production.

When you open a golem project, you’ll start with a dev-history file, which contains a series of functions that will guide you through the whole process of starting, building, and deploying your app. The newly created package contains < g class="gr_ gr_6 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="6" data-gr-id="6">an app_ui.R< g class="gr_ gr_6 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="6" data-gr-id="6"> < g class="gr_ gr_7 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="7" data-gr-id="7">< g class="gr_ gr_6 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="6" data-gr-id="6">and app_server.R< g class="gr_ gr_7 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="7" data-gr-id="7"> waiting to be filled, and a run_app() function that will launch your application. Any new module can be added < g class="gr_ gr_8 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="8" data-gr-id="8">with golem::add_module()< g class="gr_ gr_8 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="8" data-gr-id="8">, a function that creates a new file with the required skeleton for a shiny module. As I said, you don’t need to think about the technical things

You can also find a series of UI, server, and prod-related tools, functions for creating deployment scripts, and other cool stuffs. Check the README for more information.

Building Big Shiny Apps: the Tools

Package with {golem}

We believe that Shiny Apps are to be put into a package. Why? Because it allows them to be documented, < g class="gr_ gr_6 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-del replaceWithoutSep" id="6" data-gr-id="6">tested, and can be installed in several environments.

Also, think about your last Shiny app that wasn’t in a package. It’s an app.R, maybe with a folder you’re sourcing and which contains functions (let’s say in a R/ folder). Maybe you’ve written some information (let’s call it a DESCRIPTION), and some tests you’ve put in a tests/ folder. Also, as you want to be sure to do the things right, you’ve put documentation in your functions. Do you see where I’m heading? Yes, you’ve written an R package.

Git

Friends don’t let friends work on a coding project without version control.

Shiny modules

Shiny modules are crucial tools when it comes to building big shiny apps: they allow to collaborate, to split the work into pieces, they facilitate testing, and they allow implementation of new features to be made more easily.

Prototyping with {shinipsum} and {fakir}

These two tools allow you to prototype a Shiny App and to go “UI first”. Learn more :

• https://github.com/ThinkR-open/shinipsum
• https://github.com/ThinkR-open/fakir

CI and testing

Testing is central for making your application survive < g class="gr_ gr_5 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Grammar multiReplace" id="5" data-gr-id="5">on the long run. < g class="gr_ gr_32 gr-alert gr_gramm gr_inline_cards gr_run_anim Style multiReplace" id="32" data-gr-id="32">The {testthat}< g class="gr_ gr_32 gr-alert gr_gramm gr_inline_cards gr_disable_anim_appear Style multiReplace" id="32" data-gr-id="32"> package can be used to test the “business logic” side of your app, while the application features can be tested with packages like {shinytest}, or software like < g class="gr_ gr_4 gr-alert gr_spell gr_inline_cards gr_disable_anim_appear ContextualSpelling ins-del multiReplace" id="4" data-gr-id="4">Katalon.

Deploy

Docker, Shiny Proxy, RStudio Connect… Don’t just put your app on the server and let it live. Use tools that are specifically designed for deploying and scaling web applications. But this will be the topic of another blog post

Want to know more? Keep an eye on our bookdown “Building Big Shiny Apps — A Workflow“!

The post Building Big Shiny Apps — A Workflow 2/2 appeared first on (en) The R Task Force.