Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
One of the key data analysis tools that the BellKor team used to win the Netflix Prize was the Singular Value Decomposition (SVD) algorithm. As a file on disk, the Neflix Prize data (a matrix of about 480,000 members' ratings for about 18,000 movies) was about 65Gb in size — too large to be read into the standard in-memory data model of open-source R directly. But in the video below, Brian Lewis shows us how to use the sparse Matrix object in R to efficiently store the data (about 99 million actual movie ratings) and the irlba package (which features a fast and efficient SVD algorithm for big data) to perform SVD analysis on the Netflix data in R.
< embed allowfullscreen="true" allowscriptaccess="always" height="273" src="https://www.youtube.com/v/ipkuRqYT8_I?fs=1&hl=en_US" type="application/x-shockwave-flash" width="450">
Big Computing: Bryan Lewis's Vignette on IRLBA for SVD in R
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.