by Joseph Rickert When you not directly working in an industry it is often extremely difficult to get any real insight into common practices that may be blindly transparent to people who are. With some persistence though, every once in awhile you can stumble into an opportunity to see why ... [Read more...]

by Joseph Rickert Every month I look forward to getting my copy of AMSTATNEWS, the monthly magazine of the American Statistical Association, in the mail. This July, I was both pleased and bemused by ASA President Marie Davidian’s article Aren’t We Data Science?. I was pleased to see ... [Read more...]

by Joseph Rickert Early on, Revolution Analytics realized that R is more than just a tool for statistical computing — it is also the culture that has grown up around the use of the tool. The R culture is open and inclusive, competitive but also nourishing. There is a strong sense ... [Read more...]

by Joseph Rickert It is very likely that even a very casual observer what is happening in the world of R these past few months would have come across some mention of Rcpp, the R package that greatly facilitates R and C++ integration. Rcpp is hot! Over 130 R packages now ... [Read more...]

by Joseph Rickert The rxDTree() function included in the RevoScaleR package distributed with Revolution R Enterprise is an an example of a new class of algorithms that are being developed to deal with very large data sets. Although the particulars differ, what these algorithms have in common is the use ... [Read more...]

by Joseph Rickert Late last Saturday afternoon I was reading in my usual spot at the Dana Street Coffee House in Mt. View. A stranger walking by my table noticed my copy of Madsen’s Time Series Analysis (sitting there untouched again) said he needed to learn something about time ... [Read more...]

by Joseph Rickert The basic way to plot a classification or regression tree built with R’s rpart() function is just to call plot. However, in general, the results just aren’t pretty. As it turns out, for some time now there has been a better way to plot rpart() ... [Read more...]

by Joseph Rickert Quandl.com, the open source website for financial data, made rapid progress earlier this year in becoming an R friendly source for financial time series data. Tammer Kamel, Quandl’s founder introduced the site on Revolutions blog in late February as a “search engine” for numerical data ... [Read more...]

by Joseph Rickert Random number generation is fundamental to doing computational statistics. As you might expect, R is very rich in random number resources. The R base code provides several high quality random number generators including: Wichmann-Hill, Marsaglia-Multicarry, Super-Duper, Mersenne-Twister, Knuth-TAOCP-2002 and L’Ecuyer-CMRG. (See Random for details.) And, there ... [Read more...]

by Joseph Rickert In a post last week, I offered some first impressions about R/Finance 2013. Apparently, I was way off in estimating that 30% of the attendees were academics. The R/Finance organizers were quick to point out that percentage of academics attending the conference has been a constant 10% over ... [Read more...]

by Joseph Rickert On May 10th and 11th, in honor of this being the International Year of Statistics, the Milwaukee Chapter of the American Statistical Association (MILWASA) held a workshop on cutting edge uses of R in Bioinformatics. One objective of the workshop was to show the "nuts and bolts" ... [Read more...]

by Joseph Rickert This past Saturday, the New Frontiers in Computing Conference (NFIC 2013), held at Stanford University, explored the theme: Social Network Analysis: It’s Who You Know. The speakers were a well-chosen, eclectic lot who covered a remarkable array of issues in less than a full day. Ian Hersey, ... [Read more...]

by Joseph Rickert Even a casual glance at the R Community Calendar shows an impressive amount of R user group activity throughout the world: 45 events in April and 31 scheduled so far for May. New groups formed last month in Knoxville, Tennessee (The Knoxville R User Group: KRUG) and Sheffield in ... [Read more...]

by Joseph Rickert Saturday morning I was drinking my coffee wondering how much effort goes into R worldwide. (It’s my job.) I noticed that there were 4469 packages on CRAN, and it occurred to me that tabulating the packages by publication date would give some indication of how much effort ... [Read more...]

by Joseph Rickert Baseball fans have been serious about statistics since Carl Pearson was a young man (although I doubt that Carl followed the game). It is not clear, though, exactly when baseball statisticians moved from doing descriptive stats into predictive analytics. In his book Super Crunchers, Ian Ayers credits ... [Read more...]

by Joseph Rickert In a recent blog post, Revolution's Thomas Dinsmore announced stepwise regression for big data as a new feature of Revolution R Enterprise 6.2 that is scheduled for general availability later this month. Today, I would like to provide a simple example of doing stepwise regression with rxLinMod() (the ... [Read more...]

by Joseph Rickert I am beginning to get excited about going to Spain for useR 2013 which will be held at the University of Castilla-La Mancha, so I have been using the links on the Revolution's local user directory webpage to see what the European R user groups are doing. Here ... [Read more...]

by Joseph Rickert Life decisions are usually much too complicated to be attributed to any single cause, but one important reason that I am here at Revolution today is that I ignored suggestions from well-meaning faculty back in graduate school to work more in SAS rather than doing everything in ... [Read more...]

by Joseph Rickert R user groups seem to be sprouting all over. Since last September we have noticed ten new groups worldwide: Auckland, New Zealand: Auckland-R-Users-Group (AKLRUG) had 33 people attend their March 8th meeting Chang Mai Thailand: Chang Mai is the first R user group in Thailand Durban, South Africa: ... [Read more...]

by Joseph Rickert When talking with data scientists and analysts — who are working with large scale data analytics platforms such as Hadoop — about the best way to do some sophisticated modeling task it is not uncommon for someone to say, "We have all of the data. Why not just use ... [Read more...]