What NOT to do when building a shiny app (lessons learned the hard way)

Background

Shiny is a really great tool that allows data scientists to communicate their analysis in an appealing and an effective way. However, as data scientists we are used to thinking about things a certain way. Our way of thinking, and our practices are different than these of a software developer or a DevOps.

Here are some of things I learned along the path of my Shiny app developing experiences – some things that you should and should not do.

Don’t skip the planning phase

Do a mockup, research your implementation options.

Mockup

Do a mockup, even if it’s just a piece of paper which took you 5 minutes to draw. It will be worth it.

Shiny is very tempting in the sense that once you understand the concept of reactive programming, you can go from idea to a full app in a few days work. Why invest time in preparing a mockup or planning, when you can just go ahead and do the actual thing?

My experience tells me that the app is much more successful in capturing the customer’s needs, when he’s a part of the technical planning phase (when you share your dillemas with the client). It sets expectations, frames what you can and can’t (or won’t) do for the customer, and enables you to find solutions together.

Also, when you’re looking at a mockup (even if it’s just a simple drawing or a non-interactive slide), it helps in the next stages of building the app’s UI.

Here is an example of how a mockup would look like when I’m drawing it on a piece of paper. Note how I’ve already written down the purpose of some of the elements and their expeted element ids. It helps building the UI when you’re actually looking at one of these.

Example for a mockup drawing

Be sure you can live up to your promises

This is more of a broad issue (you can say its true for anything).

In my case, in the past I promised some clients I’ll provide “realtime” dashboards. However, as it turned out, I was reading from a csv data dump which provided the data with delays going up to 15-30 minutes.

In most projects I do, 15 minutes and realtime are pretty much equivalent from a practical standpoint, but in a specific project I did recently, I had a client which wanted to check the data as it was changing minute-by-minute.

This gap in expectations caused some confusion and dissappointment. We eventually learned from this, and in the future, when realtime is a requirement, we will use a better data source (i.e., data base instead of the delayed data dump).

Don’t forget to plan your budget

Make sure you consider all the elements you need for the project. Plan the budget accordingly, and understand the ramifications of scaling the app.

For example, if you’re using shinyapps.io, get familiar with the pricing packages, figure out what will you need to provide a good SLA (relative to the number of users of the app).

Same goes for other cloud services, e.g., using a data base – how many users? how many connections? size of data base?

In most cloud providers you can also set up billing alerts which lets you know when something is exceeding a predetermined threshold.

All of these are very important when you’re building your quote, and obviously when going into production with your App.

Don’t skip testing and staging on your way to production

In software development there are various levels of environments, starting from your desktop (“local”), through development server, integration, testing, staging/acceptance, and production. See Wikipedia’s Deployment environment entry.

When building an app, make sure you go through these steps. Specifically relating to testing, staging, and production). What I found to be particularly useful is to upload the app twice (in two seperate locations/urls):

1. Deploy as a beta app (client acceptance/demo) in which I demonstrate additional features and discuss them with the client, before incorporating them into production.
2. Deploy as a production/live app.

As you iterate and improve the app, fix bugs, and add new features, you are also at the risk of breaking things. Thus, you should first update the beta app, share the new additions, and let the client experiment with the app. This way you can double check you didn’t break anything else.

Only when the client authorizes the corrections, redeploy the new app to the production.

Conclusions

As data scientists using Shiny, we’ve also become software developers. We’re developing not just for ourselves or for other useRs in our community.

With Shiny we’re building for end-users. We’re building customer facing apps, and we need to keep that in mind. We should make sure that we adopt and use best practices of software development.