Regardless of the tool you use to analyse data, you'll often have to access data living in file formats generated by other tools. The "haven" package from RStudio allows you to import and export data in SAS, SPSS and Stata formats. Version 1.0 was released on October 4, and is now available ... [Read more...]

The results of many scientific papers are wrong. There are many reasons for this, including p-hacking, publication bias, and the general inability to replicate results. But there's another, more mundane cause: incorrect calculation of p-values in statistical tests. This could be caused by simple transcription errors when plugging numbers into ... [Read more...]

It's been a long time since I watched the The Simpsons, but I was always under the impression that Bart was the primary character. Perhaps it was all the Do the Bartman and "Cowabunga!" nonsense from the 90s. Anyway, data scientist Todd W Schneider used R to analyze the scripts ... [Read more...]

Two groups are making and impact in improving the gender diversity of R users worldwide. The R-Ladies organization is creating chapters worldwide to facilitate female R programmers meeting and working together, and the Taskforce on Women in the R Community is working to improve the participation and experience of women ... [Read more...]

I just got back from Atlanta, the host of the Microsoft Machine Learning and Data Science Summit. This was the first year for this new conference, and it was a blast: the energy from the 1,000 attendees was palpable. I covered Joseph Sirosh's keynote presentation yesterday, but today I wanted to ... [Read more...]

In Joseph Sirosh's keynote presentation at the Data Science Summit on Monday, Wee Hyong Took demonstrated using R in SQL Server 2016 to detect fraud in real-time credit card transactions at a rate of 1 million transactions per second. The demo (which starts at the 17:00 minute mark) used a gradient-boosted tree model ... [Read more...]

At the 2016 EARL London conference senior data-visualisation journalist John Burn-Murdoch, described how the Financial Times uses R to produce high-quality, striking data visualisations. Until recently, charts were the realm of an information designer using tools like Adobe Illustrator: the output was beautiful, but the process was a long and winding ... [Read more...]

Slides have now been posted for many of the talks given at the recent Effective Applications of the R Language (London) conference, and I thought I'd highlight a few that featured Microsoft R. Chris Cole manages the deployment of R at Investec, supporting investment and risk teams worldwide. Despite some ... [Read more...]

Hadley Wickham, co-author (with Garrett Grolemund) of R for Data Science and RStudio's Chief Scientist, has focused much of his R package development on the un-sexy but critically important part of the data science process: data management. In the Tidy Tools Manifesto, he proposes four basic principles for any computer ... [Read more...]

The Linux edition of the Data Science Virtual Machine on Microsoft Azure was recently upgraded. The Linux DSVM includes Microsoft R, Anaconda Python, Jupyter, CNTK and many other data science and machine learning tools, new or upgraded for this release. This eWeek story gives an overview of the improvements, but ... [Read more...]

Today is Talk Like A Pirate Day, the perfect day to learn R, the programming language of pirates (arrr, matey!). If you have two-and-a-bit hours to spare, Nathaniel Phillips has created a video tutorial YaRrr! The Pirate's Guide to R which will take you through the basics: installation, basic R ... [Read more...]

The Mango Solutions team have done it again: another excellent Effective Applications of R (EARL) conference just wrapped up here in London. The conference was attended by almost 400 R users from companies all around the world, and was a really fun experience. I was honored to deliver a keynote presentation, ... [Read more...]

O'Reilly has released the results of the 2016 Data Science Salary Survey. This survey is based on data from over 900 respondents to a 64-question survey about data-related tasks, tools, and the salary they receive from doing/using them. The median salary reported in the survey was US$87,000; amongst data scientists in ... [Read more...]

The R Consortium, in its most recent funding round, awarded a grant of $10,000 to The R Documentation Task Force, whose mission is to design and build the next generation R documentation system. (Microsoft is a Platinum Member of the R Consortium.) The task force has the support and participation of ... [Read more...]

In case you missed them, here are some articles from August of particular interest to R users. An amusing short video extols the benefits of reproducible research with R. A guide to implementing a churn model for mobile phone customers with Microsoft R Services. Computerworld's Sharon Machlis presents 5 data visualizations ... [Read more...]

If you want to build an application using R that serves many users simultaneously, you're going to need to be able to run a lot of R sessions simultaneously. If you want R to run in the cloud, you can publish R functions as a Web service (and you can ... [Read more...]

Microsoft R Server extends the base R language and Microsoft R Open with big-data capabilities. Specifically, it adds the RevoScaleR package, which creates an out-of-memory "CDF" data structure (so you can process data larger than available RAM), and algorithms that allow you to perform computations on such data using parallel ... [Read more...]

Microsoft R Open 3.3.1, our enhanced disstribution of open source R, is now available for download for Windows, Mac, and Linux. This update upgrades the R langauge engine to version 3.3.1, streamlines the installation process, and bundles some additional packages for parallel programming. R version 3.3.1 fixes a few rarely-encountered bugs, for example ... [Read more...]

Power BI, Microsoft's data visualization and reporting platform, has made great strides in the past year integrating the R language. This Computerworld article describes the recent advances with Power BI and R. In short, you can: import data into Power BI by using an R script cleanse and transform other ... [Read more...]

I'm excited to share that one of my data science heroes will be a presenter at the Microsoft Data Science Summit in Atlanta, September 26-27. Edward Tufte, the data visualization pioneer, will deliver a keynote address on the future of data analysis and the how to make more credible conclusions ... [Read more...]