Run Shiny app on a Ubuntu server on the Amazon Cloud

This guide is more for self reference than anything else. 
Since I struggled for two days trying to find all the correct setting to complete this task, gathering information from several websites, I decided to write a little guide on this blog so that if I want to do it again in the future and I do not remember anything (this happens a lot!!) at least I have something to resuscitate my memory.

I found most of the information and code I used from these websites:
http://tylerhunt.co/2014/03/amazon-web-services-rstudio/
http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/
http://www.rstudio.com/products/shiny/download-server/


Preface
First I would like to point out that this guide assumes you (I am talking to myself of the future) remember how to open an instance in the Amazon Cloud. It is not that difficult, you go to this page:
http://aws.amazon.com/ec2/

you log in (if you remember the credentials) and you should see the “Amazon Web Services” page, here you can select EC2 and launch an instance. Remember to select the correct server from the menu on the top right corner since the last time you run all the instances from Oregon, and you live in freaking Switzerland!!


Guide
1) Install Cygwin
This software is needed to communicate with the Ubuntu server.
It is important to follow the instructions on this page (http://www.howtogeek.com/howto/41560/how-to-get-ssh-command-line-access-to-windows-7-using-cygwin/) to install the software correctly.
In particular during the installation process a “select packages” windows appears where we need to select openssh and click on “skip”, until there is a cross on the column bin.

When Cygwin is installed we need to click with the right button on the icon and select “run as administrator”, then open it.
Now we can run the following line to install ssh:

ssh-host-config

During the process several questions will be asked, the following answers apply:
– Should privilege separation be used? YES
– New local account sshd? YES
– Run ssh as a service? YES
– Enter a value for daemon:  ntsec
– Do you want to use a different name? NO
– Create a new privilege account user? YES  -> then insert a password


After the installation we need to insert the following line to start the sshd service:

net start sshd

Then this line to configure the service:

ssh-user-config

Again it will ask a series of questions. There is a difference between the new version and what is written on the website.
Now it asks only about an SSH2 RSA identity file to be created, the answer is YES.
Then it asks other two questions regarding DSA files and another thing, the answers here are two NO.



2) Connect to the Amazon Server
Open Cygwin.
Go to the folder where the .pem file is saved, using the following line:

cd D:/<folder>/<folder>



NOTE:
Cygwin does not like folder names with spaces!


Now we need to be sure that the .pem key will not be publicly available using the following line

chmod 400 <NAME>.pem

and then we can connect to the ubuntu server using the following line:

ssh -i <NAME>.pem ubuntu@<PUBLIC IP>

These information are provided in Amazon if we click on “Connect” once the instance has properly been launched.
Once we are in we can installing R and Shiny.


3) Install R and Shiny
The first thing to do is set up the root user with the following line:

sudo passwd root

The system will ask to input a password.
Then we can log in using the following line:

su

Now we are logged in as root users.

Now we need to update everything with the following:

apt-get update

At this point we can install R with the following line:

apt-get install r-base


NOTE:
It may be that during the installation process an older version of R is installed and this may create problems with some packages.
To solve this problem we need to modify the file sources.list located in /etc/apt/sources.list
We can do this by using WinSCP, but first we need to be sure that we have access to the folder.
We should run the following two lines:

cd etc/

chmod 777 apt

This gives us access to modify the files in the folder apt via WinSCP (see point 4).
This line of code gives indiscriminate access to the folder, so it is not super secure.

Now we can connect and add the following line at the end:

deb http://cran.stat.ucla.edu/bin/linux/ubuntu trusty/

Then we need to first remove the old version of R using:

apt-get remove r-base

or

apt-get remove r-base-dev

Then we need to run once again both the update and the installation calls.



We can check the amount of disk space left on the server using the following command:

df -h


Then we can start R just by typing R in the console.
At this point we need to install all the packages we would need to run shiny using standard R code:

install.packages(“raster”)


Now we can exit from R with q() and install shiny suing the following line in ubuntu:

sudo su – -c “R -e “install.packages(‘shiny’, repos=’http://cran.rstudio.com/’)””

Now we need to install gdebi with the following lines (check here for any update:http://www.rstudio.com/products/shiny/download-server/):

apt-get install gdebi-core
wget http://download3.rstudio.org/ubuntu-12.04/x86_64/shiny-server-1.3.0.403-amd64.deb
gdebi shiny-server-1.3.0.403-amd64.deb



4) Transfer file between windows and Ubuntu
We can use WinSCP (http://winscp.net/eng/index.php) for this task.

First of all we need to import the .pem file that is need for the authentication.
From the “New Site” window we can go to “advanced”, then click on “SSH -> Authentication”.
From the “Private key file” we can browse and open the .pem file. We need to transform it into a ppk file but we can do that using the default settings.
We can just click on “Save private key” to save the ppk file and then import it again on the same field, and click OK.

Now the Host name is the name of the instance, for example:
ec2-xx-xx-xxx-xxx.eu-west-1.compute.amazonaws.com

The user name is ubuntu and the password is left blank. The protocol is SFTP and the port is 22.

The shiny server is located in the folder /srv/shiny-server

We need to give WinSCP access to this folder using again the command

cd srv
chmod 777 shiny-server





5) Transfer shiny app files on the server
In WinSCP open the folder /srv/shiny-server and create a new folder with the name of your shiny app.
Then transfer the files from your PC to the folder.
Remember to change the address of the files or working directory in the R script with the new links.


6) Allow the port 3838 to access the web
To do this we need to change the rule in the “security groups” menu.
This menu is visible in the main window where all your instances are shown. However, do not access the menu from the panel of the left, that area may be a bit confusing.
Instead select the instance in which you need to add the rule, look at the window at the bottom of the page (the one that shows the name of the instance) and click on the name in light blue near “security group”.
This will open the security group menu specific for the instance you selected.
Here we need to add a rule by clicking on “add rule”, select custom TCP, write the port number 3838 in the correct area and the select “anywhere” in the IP section.

Now if you go to the following page you should see the app:
<PUBLIC IP>:3838/<FOLDER>


7) Stop, start and restart Shiny server

sudo start shiny-server

sudo stop shiny-server

sudo restart shiny-server


8) Installing rgdal
This package require some tweaks before its installations. On this site I found what I needed: http://askubuntu.com/questions/206593/how-to-install-rgdal-on-ubuntu-12-10

Bascially from the Ubuntu console just run these three lines of code:
sudo apt-get install aptitude
sudo aptitude install libgdal-dev
sudo aptitude install libproj-dev

Then go back to R and install rgdal normally (with install.packages)