Data frames are such a straightforward and essential element of R that it’s easy to lose sight of some of their peculiarities. Last week, I developed some code which would tear apart some data frames and create new ones based on columns specified by the user. This would allow ... [Read more...]

Everyone loves to aggregate data. Everyone loves to create new columns based on other columns. Everyone hates to do the same thing twice. In my continuing work on multilevel view of loss reserving, I reached a point where I realized that I needed a robust mechanism to aggregate computed columns. ... [Read more...]

I love the syntax of calls to lm and ggplot, wherein the dataframe is specified as a variable and specific columns are referenced as though they were separate variables. While developing some of my functions, I’d wanted to introduce something similar. I often find that I have a single ... [Read more...]

Of all the things I dislike about R, one of the biggest is the fact that you can declare a function within the list of arguments to another function. I’ve gotten over it for very minor operations needed by things like lapply, but it can drive me bonkers elsewhere. ... [Read more...]

  Anytime I get a new hammer, I waste no time in trying to find something to bash with it. Prior to last year, I wouldn’t have known what a cluster was, other than the first half of a slang term used to describe a poor decision-making process. Now I’... [Read more...]

So, you’ve written some code and you use it routinely. Now you’ve written some code and you’d like to use version control to ensure that development continues in a robust fashion. You do that and you use Github or something so that not only are changes tracked, ... [Read more...]

Having posted some code to Git a few days ago and having been wholly dissatisfied with it, I began to do what I often do with code I don’t like. I started re-writing it bigger and weirder and more philosophically pure. Part of this search for Platonic code lead ... [Read more...]

I started using Git some time ago, but mostly for local work files. Today, I finally sync’ed up a repository for loss reserving analysis. It may be found here: https://github.com/PirateGrunt/MRMR MRMR stands for Multivariate Regression Model for Reserves. When pronounced “Mister Mister” it also sounds ... [Read more...]

Last year, I spent a morning trying to return to first principles when modeling loss reserves. (Brief aside to non-actuaries: a loss reserve is the financial provision set aside to pay for claims which have either not yet settled, or have not yet been reported. If that doesn’t sound ... [Read more...]

I’ve made some revisions and simplifications to the code to compile NFL data. It’s now all out on Github for anyone to play with in advance of the Superbowl. In the meantime, here’s a lovely picture comparing every team’s offense- as measured by total offensive yards- ... [Read more...]

So as I sink deeper into the second level of R enlightenment, one thing troubled me. “lapply” is fine for looping over a single vector of elements, but it doesn’t do a nested loop structure. These tend to be pretty ubiquitous for me. I’m forever doing the same ... [Read more...]

I love it when things are easy. I love it so much that I’ll spend a great deal of time and effort to keep things simple. At the same time, though, I think there’s some value in expending effort in pursuit of something. If you want to understand ... [Read more...]

This is probably a slightly useless post, but a bit of fun all the same. If nothing else, it allows me to take a stab at learning a bit more about logistic regression. I’m still trying to unravel the mystery of why the Bears lost to the Vikings two ... [Read more...]

One of the first weird things to get used to in R is unlearning some of the things that you think you know. As often happens, this reminds me of a quote I once read about Zen, which went about like this (I’m paraphrasing), “When I knew nothing of ... [Read more...]

I’ve been doing a lot of linear modeling this year. That’s not much different than any ordinary year, but now I’m doing it in R. I had spent a bit of time in recent years trying to look at loss reserving as a multivariate regression. Excel is ... [Read more...]

I’ve spent a good deal of 2012 constructing a data warehouse to manage all the various data elements that my company has. Although we’re a small enterprise, the richness and complexity of the information is rather high. Moreover, as a data-driven organization, there’s a strong impetus to construct ... [Read more...]

So I tidied the code up a bit from last time; no more for loop. Actually, I tidied it up a lot. My goal had been to arrange the data in such a way that I could get a simple moving average of the score difference for each team. That ... [Read more...]

I’ve made the acquaintance of a group of data analysts here in the triangle and have agreed to arrange an outing to the Durham Bulls minor league baseball team. Because it’s for stat nerds and because I was curious, I went looking for some baseball data to analyze. ... [Read more...]