Setting up a parallel computing cluster for R with OpenSSH and doSNOW
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Responding to yesterday's post which included an aside on using parallel processing for by-group computations in R, reader Christian Gunning mused about the possibility of using doSNOW on his network, with OpenSSH to manage the authentication:
I sit on a fast campus network and have at least 10 remote cores available that I could farm out for big jobs. the SSHcluster method would require minimal invasion on those machines — just ability to execute ssh and Rscript on the remote machines — not even login privileges are required!
Well, Christian developed that idea into a step-by-step guide for setting up a multi-node cluster for parallel programming using foreach and doSNOW. Check out the full guide (including both R and shell configuration code) at the link below.
Life In Code: snow and ssh — secure inter-machine parallelism with R
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.