Not Hotdog: A Shiny app using the Custom Vision API

I had a great time at the EARL Conference in London last week, and as always came away invigorated by all of the applications of R that were presented there. I'll do a full writeup of the conference later this week, but in the meantime I wanted to share the materials from my own presentation there, "Not Hotdog: Image Recognition with R and the Custom Vision API". I've embedded the slides below:

In the presentation, I showed how easy it is to build a Shiny app to perform image recognition with the Custom Vision API in Azure. I've provided all of the R code behind the app in this GitHub repository, and thanks to the httr package it takes just a few lines of R code to interface with the API to upload images, train a transfer learning model, and classify new images. It's also handy to be able to use the customvision.ai web portal to review your tagged images and model performance. 

This was my first time building a Shiny app, and it was really easy to learn how to do it thanks to the RStudio tutorials. One feature that proved particularly useful was reactive expressions, which would cache the results of the prediction and prevent a call out to the API every time I touched the "threshold" slider. The prediction API is rate-limited, and this way it was only called when the user actually changes the image URL, a relatively rare event. In all, it took me less than a day to learn Shiny and create the application below. Here it is correctly classifying this image of a hotdog (source), and correctly identifying that this hot dog (source) is not a hotdog.

If you want to try this out yourself, all you'll need is an Azure subscription (new subscribers can also get $200 free credits) and the R code in the repository below.

Github (revodavid): Not Hotdog vision recognition using R and Custom Vision API