Here is a simple application to transform text into a beautiful word cloud, Text Mining to WordCloud. The purpose is to find out the highest frequency word in a certain text. It is an app built with R language, the source code is attached at the end of... [Read more...]

Next week I’ll present a glimpse of R and ggplot2 graphics at VUW. This is a MESA seminar on ‘Data analysis and plotting with free and open source tools’ where we’ll present spreadsheet alternatives based on gnuplot, Python, an... [Read more...]

Being freshly elected ASA Fellow (yay!), I just received the list of 2012 ASA Fellows. Among whose, let me mention Sudipto Banerjee, University of Minnesota, Minneapolis, Minnesota, elected “For theoretical, methodological and applied research in spatiotemporal statistical modeling, especially as applied to problems in environmetrics, ecology, occupational health, agriculture and economics, ... [Read more...]

Kevin Quealy, graphics editor at the New York Times, has published another fascinating behind-the-scenes look at how the Times creates data visualizations for print and online. In his latest post, he looks at how a visualization of the Yankee's Mariano Rivera performance compared to other Major League Baseball pitchers was ... [Read more...]

Turns out that Megan would have never gotten a callback for an audition. (Via Ben Schmidt.) [Read more...]

I can’t believe I didn’t learn how to do it earlier, but I never knew how to accurately copy tables from excel that had text with spaces in them, and paste into a data frame in R without generating confusion … Continue reading → [Read more...]

In case you missed them, here are some articles from April of particular interest to R users. Information Age published a feature article on R, describing how new graduates are driving adoption of R in industry. Bob Muenchen has updated his list of R package equivalents to SAS and SPSS ... [Read more...]

As I’ve discussed here before, there is a debate raging (ok, maybe not raging) about terms such as “data science”, “analytics”, “data mining”, and “big data”. What do they mean, how do they overlap, and perhaps most importantly, who are the people who work in these fields? Along with ... [Read more...]

In part 2, we saw that adding a volatility filter to a single instrument test did little to improve performance or risk adjusted returns. How will the volatility filter impact a multiple instrument portfolio? In part 3 of the follow up, I will evaluate the impact of the volatility filter on a ... [Read more...]

In yesterday's webinar, Revolution Analytics CTO David Champagne demonstrated how to integrate statistical graphics and analytic computations created using R software with a variety of third-party applications. In each case Revolution R Enterprise Server is running as a compute server to the client application, with R scripts launched on each ... [Read more...]

Accounting for temporal dependence in econometric analysis is important, as the presence of temporal dependence violates the assumption that observations are independent units. Historically, much less attention has been paid to correcting for spatial dependence, which, if present, also violates this independence assumption. The comparability of temporal and spatial dependence ... [Read more...]

by Yanchang Zhao, RDataMining.com My book in draft titled “R and Data Mining: Examples and Case Studies” is now available on CRAN at http://cran.r-project.org/other-docs.html. It is scheduled to be published by Elsevier in late 2012. Its latest version can be … Continue reading → [Read more...]

  I can never seem to get exactly what I want from an R text editor. Let me correct that, I can never seem to get exactly what I want from an R text editor on a MAC. I used to use Tinn-R  which met most  my needs: Free,lightweight with ... [Read more...]

The more I’ve worked with my really large data set, the more cumbersome the work has become to my work computer.  Keep in mind I’ve got a quad core with 8 gigs of RAM.  With growing irritation at how slow … Continue reading → [Read more...]

There are quite a few books out now on “data science”. I’ve picked out three that I think are the best place to start for computational journalists. First is Machine Learning for Hackers, by Drew Conway and John Myles White. The autho... [Read more...]

Foursquare, the mobile location-sharing app (of which I'm a big fan), has an excellent recommondation system. Based on your recent checkins, places your friends found popular, and even the time of day, Foursquare Explore will recommend a great place for a sushi lunch, or the best place to buy new ... [Read more...]

The recent Hack/Reduce hackathon in Montreal was a tonne of fun. Our team tackled a data set of consisting of Bixi (Montreal’s bicycle share system) station states at one minute temporal resolution. We used Hadoop and mapreduce to pull out some features of user behaviours. One of the ... [Read more...]

Diversification is hard to find nowadays because financial markets are becoming increasingly correlated. I found a good visually presentation of Cross Sectional Correlation of stocks in the S&P 500 index in the Trading correlation by D. Varadi and C. Rittenhouse article. Let’s compute and plot the average correlation among ... [Read more...]

We have now completed our revision of the paper Relevant statistics for Bayesian model choice, written with Judith Rousseau, Jean-Michel Marin, and Natesh Pillai. It has been resubmitted to Series B and reposted on arXiv. The major change in the paper is the inclusion of a check about the relevance ... [Read more...]

The story about the great work that SUNY Buffalo has been doing to find a cure for Multiple Sclerosis with Revolution R Enterprise and IBM Netezza has generated a lot of attention, with stories in Forbes, InformationWeek and eWeek (amongst others). To continue the discussion, IBM has put together a ... [Read more...]