Update Existing Shiny Apps in ShinyProxy

ShinyProxy is one of the best self-hosting options out there when you need enterprise features, like authentication and app-level authorization, for your app. In this post, I explain how to manage existing applications. All this without restarting your ShinyProxy server and thus keeping all your users connected. There are a few important considerations to remember when working with Docker images.

You can find the code for this tutorial in the analythium/shiny-correlation GitHub repository:

GitHub – analythium/shiny-correlation: Containerized Shiny app with multiple versions

Containerized Shiny app with multiple versions. Contribute to analythium/shiny-correlation development by creating an account on GitHub.

GitHubanalythium

What to consider when updating apps

Remember how the apps are listed in the application.yml file in the /etc/shinyproxy folder:

  - id: 02_hello
    display-name: Demo Shiny App
    description: App with sliders and file upload
    container-cmd: ["R", "-e", "shiny::runApp('/home/app')"]
    container-image: analythium/shinyproxy-demo:latest
    access-groups: [admins]

There are two ways of updating the apps in ShinyProxy. You can update only the Docker image. This means that the container-image tag and everything else in the config YAML file stays the same. Because the application.yml file remains the same, there is no need to restart the ShinyProxy service on the server.

If you change anything in the application.yml configuration file, you'll have to restart the ShinyProxy service for these changes to take effect. This might include replacing a container-image tag, adding new apps, or touching any of the general configurations.

Deploying and managing ShinyProxy can get complex when many apps are used, especially when the configuration of ShinyProxy is often updated. When restarting a running ShinyProxy instance (in order to update its configuration), users will face a disconnect from their running applications. – ShinyProxy Docs

Restarting ShinyProxy would mean that all current users would lose their Shiny sessions and logged-in users would be logged out. This is a disruption one can avoid by managing the Docker images carefully. You can also schedule the updates to times when there are no users logged in, or when the disruption is minimal and advertised to users in advance.

Let's focus on the first scenario. The rest of the post explains how you can manage your image tags to minimize disruption to your users.

Add an app to ShinyProxy

The analythium/shiny-correlation GitHub repository contains a simple Shiny app. This app is a bivariate version of the Hello Shiny histogram example. We will build this app locally, then deploy it to the ShinyProxy server.

Build the Docker image

Let's build the Docker image without a tag, using the -f flag to specify which Dockerfile to use:

docker build -f Dockerfile-v1 -t analythium/correlation .

The build process should be pretty fast because I specified a parent image that already contains all the dependencies. Here is the Dockerfile-v1:

FROM analythium/shinyproxy-demo:latest
RUN rm -rf /home/app/*
COPY ./app-v1.R ./app.R

If you list the Docker images with docker image ls, you see something interesting. The image has a tag called :latest, we'll come back to this in a bit:

REPOSITORY                   TAG                 IMAGE ID
analythium/correlation       latest              90732201668c

You can test the image by running the docker run -p 4000:3838 analythium/correlation command and visiting http://localhost:4000.

Change the sample size and the correlation and watch how the 2-dimensional estimates of the intensity are changing:

Add the app to ShinyProxy

Follow the ShinyProxy setup instructions or use the ShinyProxy 1-click app from the Digitalocean Marketplace to set up your ShinyProxy server.

Push the image to the registry (docker push analythium/correlation:latest), add it to the ShinyProxy configuration, pull the image on the host machine, and restart the ShinyProxy service with service shinyproxy restart.

Update an existing app

When the Docker image itself changes, but its name (including the tag) stays the same, you won't have to restart ShinyProxy. This can happen when you push a new version of the Docker image to your Docker registry, e.g. after updating the analythium/correlation image.

Update the Docker image

The version 2 update of the Shiny app includes a 3D representation of the distribution using the wonderful rgl R package. Say, we want to tag the new image as :v2 because we'd like to keep both versions. We use Dockerfile-v2:

FROM analythium/shinyproxy-demo:latest
USER root
RUN install2.r -r http://cran.rstudio.com/ rgl
RUN rm -rf /home/app/*
COPY ./app-v2.R ./app.R
USER app

This Dockerfile copies version 2 of the Shiny app into the image and installs the rgl package.

Let's build the new image:

docker build -f Dockerfile-v2 -t analythium/correlation:v2 .

Now print out the list of Docker images with docker image ls:

REPOSITORY                   TAG                 IMAGE ID
analythium/correlation       v2                  9d98df8ea853
analythium/correlation       latest              90732201668c

We have a new image tagged as :v2 but the previous image stayed the :latest. This is counter-intuitive behaviour because we would think that the :latest tag would point to the image that was built last. Instead, it means "the last build/tag that ran without a specific tag/version specified".

If you push a new image with a tag which is neither empty nor ‘latest’, :latest will not be affected or created. – Vladislav Supalov

Image tagging and versioning

You need to be careful when omitting the image tag. Here is what you can do to version the images and to also keep the :latest tag consistent with the intuitive notion of the latest image.

Let's tag the previously created (version 1) image as :v1 using the docker tag command:

docker tag analythium/correlation analythium/correlation:v1

List and verify, :latest is now an alias for :v1 because the image IDs match:

REPOSITORY                   TAG                 IMAGE ID
analythium/correlation       v2                  9d98df8ea853
analythium/correlation       latest              90732201668c
analythium/correlation       v1                  90732201668c

Now rebuild the image without the tag (this should be quick due to caching), this will make the new image the :latest. Then tag the latest image as the new version:

docker build -f Dockerfile-v2 -t analythium/correlation .
docker tag analythium/correlation analythium/correlation:v2

List and verify again, the :latest image ID matches :v2:

REPOSITORY                   TAG                 IMAGE ID
analythium/correlation       latest              9d98df8ea853
analythium/correlation       v2                  9d98df8ea853
analythium/correlation       v1                  90732201668c

Test the new image with docker run -p 4000:3838 analythium/correlation and visit http://localhost:4000/ to play with the 3D rendering of the correlated variables:

Push these images to the registry:

docker push analythium/correlation
docker push analythium/correlation:v1
docker push analythium/correlation:v2

Update the app on the ShinyProxy server

After this little Docker detour, we are on track to update the app in ShinyProxy because the latest tag will mean version 2:

1. ssh into the ShinyProxy server as the root user
2. pull the new version of the image with docker pull analythium/correlation:latest

When you update the image, there is no need to restart the Docker or ShinyProxy services. The next time a user starts the application on your ShinyProxy installation, the new image will be used to launch a container.

Versioning your images is considered good practice. It makes a lot of sense so that you can roll back changes when something unexpected happens.

If you tag your images carefully and test the apps locally, you can use the :latest image tag in your ShinyProxy configuration without much trouble. You can also roll back to :v1 if anything unexpected comes up – you'll have to change the config and restart the service in this case.

Update multiple existing apps

If you have multiple apps, you will have to pull the latest version for each. You can use a script for that, or use the following 1-liner to update all the Docker images on your host:

docker images |grep -v REPOSITORY|awk '{print $1":"$2}'|xargs -L1 docker pull

Sometimes, you might want to remove dangling images with docker image prune -f. A dangling image is one that is not tagged and is not referenced by any container, i.e. intermediate layers, etc., which might not be used by the latest images. These can accumulate over time and take up space.

Add a Cron job

You can set up a Cron job to periodically update the images on the server. Run crontab -e as root to have access to the Cron utility. Pick an editor (e.g. nano) if you haven't done so already and then add these lines to the bottom of the file and save it:

# Cleanup at 3:00am every Sunday
0 3 * * 0 docker image prune -f

# Update all images at 1:00am every day
0 1 * * * docker images |grep -v REPOSITORY|awk '{print $1":"$2}'|xargs -L1 docker pull

Check the settings using crontab -l.

Cron jobs represent a polling type of update, which means we are regularly checking for updates. On one hand, if changes to the images are infrequent, there is no need for constant polling. On the other hand, setting Cron intervals too large might lead to missing important updates.

Pick the frequency of updates with this in mind. Webhooks for existing apps and images are considered a better alternative to polling, although webhooks require a bit more work that I am not covering here.

Conclusions

Updating existing apps for your ShinyProxy server is straightforward. You are only one command away from updating all the already deployed apps at once – no disruption to your users. The next user who opens the app will see the updated version. However, be careful with tagging and versioning your images to avoid unwanted surprises.

Further reading

• The misunderstood Docker tag: latest
• Pruning unused Docker images
• Cron guide