[This article was first published on Opiate for the masses, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
tl;dr
Simple image recognition app using TensorFlow and Shiny
About
My weekend was full of deep learning and AI programming so as a milestone I made a simple image recognition app that:
Takes an image input uploaded to Shiny UI
Performs image recognition using TensorFlow
Plots detected objects and scores in wordcloud
App
This app is to demonstrate powerful image recognition functionality using TensorFlow following the first half of this tutorial.
In the backend a pretrained classify_image.py is running, with the model being pretrained by tensorflow.org.
This Python file takes a jpg/jpeg file as an input and performs image classifications.
I will then use R to handle the classification results and produce wordcloud based on detected objects and their scores.
Requirements
The app is based on R (shiny and wordcloud packages), Python 2.7 (tensorflow, six and numpy packages) and TensorFlow (Tensorflow itself and this python file).
Please make sure that you have all the above packages installed. For help installing TensorFlow this link should be helpful.
Structure
Just like a usual Shiny app, you only need two components; server.R and ui.R in it.
This is optional but you can change number of objects in the image recognition output by changing the line 63 of classify_image.py
server.R
I put comments on almost every line in server.R so you can follow the logic more easily.
ui.R
The ui.R file is rather simple:
Shiny App
That’s it!
Here is a checklist to run the app without an error.
Make sure you have all the requirements installed
You have server.R and ui.R in the same folder
You corrently set PYTHONPATH and CLASSIFYIMAGEPATH
Optionally, change num_top_predictions in classify_image.py
Upload images should be in jpg/jpeg format
I was personally impressed with what machine finds in abstract paintings or modern art ?