by Dmitry Pechyoni, Microsoft Data Scientist The New York City taxi dataset is one of the largest publicly available datasets. It has about 1.1 billion taxi rides in New York City. Previously this dataset was explored and visualized in a number of blog posts, where the authors used various technologies (e.... [Read more...]

by Joseph Rickert IBM Joins the R Consortium This past Monday at the Spark Summit in San Francisco IBM announced that it had joined the R Consortium as a "Platinum" member. This is very good news with respect to the development and growth of the R language, the health of ... [Read more...]

by Max Kuhn: Director, Nonclinical Statistics, Pfizer Many predictive and machine learning models have structural or tuning parameters that cannot be directly estimated from the data. For example, when using K-nearest neighbor model, there is no analytical estimator for K (the number of neighbors). Typically, resampling is used to get ... [Read more...]

by Joseph Rickert The model table on the caret package website lists more that 200 variations of predictive analytics models that are available withing the caret framework. All of these models may be prepared, tuned, fit and evaluated with a common set of caret functions. All on its own, the table ... [Read more...]

by John Mount Ph. D. Data Scientist at Win-Vector LLC In her series on principal components analysis for regression in R, Win-Vector LLC's Dr. Nina Zumel broke the demonstration down into the following pieces: Part 1: the proper preparation of data and use of principal components analysis (particularly for supervised learning ... [Read more...]

by Joseph Rickert R / Finance 2016 lived up to expectations and provided the quality networking and learning experience that longtime participants have come to value. Eight years is a long time for a conference to keep its sparkle and pizzazz. But, the conference organizers and the UIC have managed to create ... [Read more...]

by John Mount Ph. D. Data Scientist at Win-Vector LLC In part 2 of her series on Principal Components Regression Dr. Nina Zumel illustrates so-called y-aware techniques. These often neglected methods use the fact that for predictive modeling problems we know the dependent variable, outcome or y, so we can use ... [Read more...]

by Joseph Rickert For quite a few years now we have attempted to maintain the Revolution Analytics' Local R User Group Directory as the complete and authoritative list of R user groups. Meetup groups make this list in one of two ways: we discover the group because they have a ... [Read more...]

John Mount Ph. D. Data Scientist at Win-Vector LLC Win-Vector LLC's Dr. Nina Zumel has just started a two part series on Principal Components Regression that we think is well worth your time. You can read her article here. Principal Components Regression (PCR) is the use of Principal Components Analysis (... [Read more...]

by Joseph Rickert What makes for a good R package? With over 8,000 packages up on CRAN the quantity of packages is clearly not an issue for R users. Developing an instinct to recognize quality, however, both requires and deserves some effort. I regularly spend time on Dirk Eddelbuettel’s CRANberries ... [Read more...]

by Katherine Zhao, Hong Lu, Zhongmou Li, Data Scientists at Microsoft Bicycle rental has become popular as a convenient and environmentally friendly transportation option. Accurate estimation of bike demand at different locations and different times would help bicycle-sharing systems better meet rental demand and allocate bikes to locations. In this ... [Read more...]

by Yuzhou Song, Microsoft Data Scientist R is an open source, statistical programming language with millions of users in its community. However, a well-known weakness of R is that it is both single threaded and memory bound, which limits its ability to process big data. With Microsoft R Server (MRS), ... [Read more...]

by Joseph Rickert When I first went to grad school, the mathematicians advised me cultivate the habit of reading with a pencil. This turned into a lifelong habit and useful skill for reading all sorts of things: literature, reports and newspapers for example; not just technical papers. However, reading statistics ... [Read more...]

by Joseph Rickert Answering email queries from friends and acquaintances from around the world wanting to attend useR! 2016 has been painful. It is amazing that the conference sold out a full two months before its start, but upon reflection, not unbelievable. From its inception useR! has been an "academic" conference ... [Read more...]

by Lixun Zhang, Data Scientist at Microsoft As a data scientist, I have experience with R. Naturally, when I was first exposed to Microsoft R Open (MRO, formerly Revolution R Open) and Microsoft R Server (MRS, formerly Revolution R Enterprise), I wanted to know the answers for 3 questions: What do ... [Read more...]

by Joseph Rickert R/Finance 2016 is less than a month away and, as always, I am very much looking forward to it. In past years, I have elaborated on what puts it among my favorite conferences even though I am not a finance guy. R/Finance is small, single track ... [Read more...]

by Joseph Rickert The R Consortium, the non-profit trade organization formed under the Linux Foundation to support the R language and the R Community, is beginning to build real momentum. First of all, two new companies recently joined the Consortium: Avant which provides online personal and auto loans and Procogia, ... [Read more...]

by Joseph Rickert Basically, there are two kinds of graphics or plots you can make from a data set: (1) those that allow you to see what is going on with the data, and (2) those you make to communicate what you have found to someone else. When making the first kind, ... [Read more...]

By Srini Kumar, Director of Data Science at Microsoft Who does not hate being stopped and given a traffic ticket? Invariably, we think that something is not fair that we got it and everyone else did not. I am no different, and living in the SF Bay Area, I have ... [Read more...]

by Joseph Rickert Packages continue to flood into CRAN at a rate the challenges the sanity of anyone trying to keep up with what's new. So far this month, more than 190 packages have been added. Here is a my view of what's interesting in this March madness. The launch_tutorial() ... [Read more...]