Building Big Shiny Apps — A Workflow (1/2)

During the < g class="gr_ gr_4 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="4" data-gr-id="4">rstudio:< g class="gr_ gr_8 gr-alert gr_gramm gr_inline_cards gr_run_anim Style replaceWithoutSep" id="8" data-gr-id="8">:conf(2019L), I’ve presented an < g class="gr_ gr_6 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="6" data-gr-id="6">eposter called “Building Big Shiny Apps — A Workflow”. You can find the poster here, and this blog post is an attempt at a transcription of what I’ve been talking about while presenting the poster.

As this is a rather long topic, I’ve divided this post into two parts: this first post will talk about the background and motivation, and the second post will present a step by step workflow and the necessary tools.

Motivation

The idea behind this poster (and now this blog post) is not to talk about how to deploy and scale, but about the process of building the app. Why? A lot of blog posts and books talk about putting apps in production. Very few talks about working on building these apps. This is why I’ve chosen to talk about the process, workflow, and tools we use at ThinkR when building big Shiny Apps.

So,we’ll not talk about what to do when the app is ready, we’ll talk about how to make it ready.

About Big Shiny Apps

If you’re reading this page, chances are you already know what a Shiny App is — a web application that communicates with R, built in R, and working with R. With Shiny, almost anybody can create a prototype for a small data product in a matter of hours. And no knowledge of HTML, CSS or JavaScript is required, making it really easy to use as you can rapidly create a POC.

But what to do now that you want to build a big Shiny App?

What’s a big Shiny App?

• Well, first, one that includes several thousand lines of code (R and others).
• It’s also one that is potentially developed by several coders, working on the same application at the same time.
• It’s an application where scaling matters.
• Maintainability and ease of upgrading are important.
• In many cases, Shiny Apps in production are not used by “tech literate” users.
• People rely on this application for making real-world decisions, with real consequences — just as phrased by Joe Cheng with his definition of what “in production” means:

Building a Big Shiny App, the Challenges

Finding a good UI (and stick with it)

Choosing a UI is hard — we have a natural tendency, as coders, to be focused on the backend, i.e the algorithmic part of the application. But let’s state the truth: no matter how complex and innovative your backend is, your application is bad is your UI is bad. That’s the hard truth. If people can’t understand how to use your application, your application doesn’t work. No matter how incredible the backend is.

So try to find a simple and efficient UI (I’m personally a big fan of minimalistic UI). One that people can understand and use in a matter of seconds. Don’t implement features or visual elements that are not actually needed, just “in case”. And spend time working on that UI, really thinking about what visual elements you are implementing.

This part is a crucial part, as it will influence the rest of the work — a big app means numerous features, and it can be hard to find a way to organise all these features in an understandable, easy to use user interface.

Working as a team

Big Shiny Apps usually means that several peoples will work on the application. For example, at < g class="gr_ gr_5 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="5" data-gr-id="5">ThinkR, 3 to 4 peoples usually work on the application. So, how do we organize that?

From the tools point of view:

• Use version control (not sure I have to expand on that topic )
• Think of your shiny app as a tree, and divide it as much as possible into little pieces. Then, create one Shiny module by piece. This allows you to split the work, and also to have smaller files — it’s easier to work on 40 files of 200 lines than on one big app.R file.

Here is, for example, the division of an app into modules and sub modules:

From the organisational point of view

• Define one person who is in charge of having the big picture. This person will kick off the project, and write the skeleton of the app, with the good modules and files structure. This person will also be in charge of accepting new merge requests from other developers, and to orchestrate the master and dev branches.
• List the tasks, and open one issue for each task on your version control system. Each issue will be solved in a separate branch.
• Finally, assign one module to one developer — if it seems that working on one module is a two-person job, divide again into two other submodules. This is a relatively complex task, as the output of one module influences the input of another, so be sure to assign them well. This division into modules allow working simultaneously on the same app without interfering with other people’s code — as each developer is working on their own modules, collaboration is < g class="gr_ gr_415 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar only-ins doubleReplace replaceWithoutSep" id="415" data-gr-id="415">way simpler.

Making the app production ready

This includes two things: scaling and maintaining. As said in the disclaimer, I won’t expand on the topic of scaling, as many have written about that, but here is one piece of advice: make the R process running the app do as less as possible, and in particular prevent it from doing what it’s not supposed to do. Which includes: use JavaScript so that the client browser renders things (instead of making R do the work — and don’t worry, basic JS is easy to learn ), use parallelization and async, and if possible, make the heavy lifting be done outside the R session running the app.

Maintainance, on the other, is something to think about from the beginning. It includes being able to ensure that the application will work < g class="gr_ gr_4 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar multiReplace" id="4" data-gr-id="4">on the long < g class="gr_ gr_5 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-del replaceWithoutSep" id="5" data-gr-id="5">run, and that new features can be easily implemented.

• Working in the long run: separate the code with “business logic” (aka the data manipulation and the algorithm, that can work outside the context of the app) from the code building the application. That way, you can write regression tests for these functions to ensure they are stable.
• Implement new elements: as we are working with modules, it’s easy to insert new elements inside the global application.

That’s it for today! Stay tuned for the second part of this post that will describe the workflow in details, and talk about the tools to use

The post Building Big Shiny Apps — A Workflow (1/2) appeared first on (en) The R Task Force.