I’ve just uploaded version 1.1 of  the package RghcnV3 to Cran. I’ve made a few changes that should make it easier for some folks to use. First I removed the requirement for rgdal. At the present time “rgdal” is not required. On the MAC installing it can be a ... [Read more...]

With RTextTools now released and the feedback rolling in, the development team is getting the ball rolling on the help documentation for the library. Currently, you cannot access help files about the library or its functions from within R. However, we do offer a draft of a quick start guide ... [Read more...]

As you can tell from the content on this blog, I am a really big fan of statistical analysis and the NHL.  I haven’t blogged in some time simply because I have been deeply engrossed by the 2011 playoffs, where … Continue reading → [Read more...]

I am down to the last strokes of creating a package for downloading, importing and analyzing GHCN (Global Historical Climate Network0 Version 3.  There  were a few things holding me up, but  I’ve manged to handle all of them except the last one.  Let’s spend a little time looking ... [Read more...]

It’s been a long journey and there are some people to thank for helping me along the way. Steve McIntyre, Ron Broberg, Jeff Id, Ryan ODonnell, RomanM, Nick Stokes, Robert Hijmans, Gabor  Grothendieck, Hadley Wickham, David Winsemius, and countless others on the R Help list. The Package is done. ... [Read more...]

The biggest complaint we had during the installation process was that Xcode (account required) and Rtools were required for MacOS X and Windows. Today we released universal binaries (PPC/i386/x86_64) for MacOS 10.5+ as well as binaries (i386/x86_64) for Windows. This addition wil [...] [Read more...]

When we first wrote RTextTools, we opted to use RWeka for boosting and bagging algorithms for lack of a better alternative. We've discovered that this leads to all sorts of ugly rJava installation issues across platforms and prevents our users from getting started quickly. Recently, we've stumbled upon two excellent ... [Read more...]

I did the Two Castles Run today; it’s a 10km race between Warwick and Kenilworth castles. The organizers were very quick to put the results online and even went the extra mile of offering them as a CSV file. It … Continue reading → [Read more...]

After several weeks trying to find the source of a bug in the maximum entropy library when compiling on Windows, Dirk Eddelbuttel pointed me in the right direction to resolve the issue. Although it required a re-write of the library using the new Rcpp API, maximum ent [...] [Read more...]

Thanks to this great post http://www.imachordata.com/?p=730 we can now put multiple plots on a display with ggplot2. This provides somewhat similar functionality to ‘par(mfrow=c(x,y))’ which would allow multiple plots with the base plot function. gridExtra doesn’t have quite the same level ... [Read more...]

There are any number of short tutorials on making add on R packages on your Windows machine. This is yet another version of that process. I’ve explained what I did in 10 easy steps on the pages, but I’ll give a brief overview here. In the first step I ... [Read more...]

Recently I was asked to help create random IDs for someone. At first I thought, ‘Ah yup, 1:x (1,2,3, …,x), job done’. Then I thought that there had to be a R function/package to create better looking IDs, to which I didn’t find one, if there is, please let ... [Read more...]

In preparation for The 4th Annual Conference of the Comparative Policy Agendas Project in Catania, Sicily, our development team has been busy drafting the documentation for RTextTools. In addition to standard documentation of functions, we want to provide quick-start guides, sample datasets, example scripts, and [...] [Read more...]

One key limiting factor for automated text classification is memory consumption. As you accumulate more news articles, bills, and legal opinions, the term-document matrices used to represent the data grow quickly. RTextTools provides two algorithms, support vector machines and maximum entropy, that can handle large datasets with very little memory. ... [Read more...]

I’ve received one those FW:…:FW emails yesterday with the following text: 2011 is an unusual year. Add the last two digits of your birth year to the age you will turn on your birthday this year and you’ll get 111! … Continue reading → [Read more...]

In creating the numbers tool I wanted to be able to do two things, 1) obtain information about what source did by matching the numeric literals it contained against a database of ‘interesting’ values (now with over 14,000 entries) and 2) flag possible incorrect numeric literals (e.g., 3.1459265 when 3.14159265 had been intended in ... [Read more...]

There are lots of R documents and over 3000 R packages. Which documents to start with and which R packages to use are questions faced by many people. If you are new to R, an Introduction to R can be … Continue reading → [Read more...]

Right now our development team is busy preparing a conference release of RTextTools for The 4th Annual Conference of the Comparative Policy Agendas Project at the University of Catania in Sicily. One of the key issues we've had thus far is memory consumption with very large datasets.In the past ... [Read more...]

My thanks to those who emailed comments and suggestions for my ‘mfx’ function, I’m happy that I could fill a void for some people. I also received a request/suggestion from Tony Cookson, along with a helpful fix for a bug in the code, to include an option that ... [Read more...]

Although this blog’s primary focus is time series, one feature I missed from Stata was the simple marginal effects command, ‘mfx compute’, for cross-sectional work, and I could not find an adequate replacement in R. To bridge this gap, I’ve written a (rather messy) R function to produce ... [Read more...]