So, an exploration of how tennis tournament seeding and bracketing impacts on the end result has percolated to the top of my to-do list, inspired by the Melbourne Open currently in play. This is the first of what will probably be two posts on this top... [Read more...]

The New York City Taxi & Limousine Commission’s open data of taxi trip records is rightly a go-to test piece for analytical methods for largish data. Check out, for example: The benchmark of interesting analysis is set by Todd W Schneider’s... [Read more...]

Even organisations and people that use a statistical package for hard-core data and econometric analysis might use spreadsheets for financial and economic scenario and evaluation models. There’s something about the ad hoc nature of financial models i... [Read more...]

An interesting big data thought experiment The other day on Twitter I saw someone referencing a paper or a seminar or something that was reported to examine the following situation: if you have an urn with a million balls in it of two colours (say red... [Read more...]

I recently did some pro bono work for Gun Control NZ reviewing the analysis by a market research firm of the survey that led to this media release: “Most New Zealanders back stronger gun laws”. The analysis all checked out ok. The task at that time... [Read more...]

Did the average rate of Australian mass-shooting decline after 1996, or was the drop just chance? I recently came across this letter to the Annals of Internal Medicine by Simon Chapman, Michael Stewart, Philip Alpers and Michael Jones: Fatal Firearm Incidents Before and After Australia’s 1996 National Firearms Agreement Banning Semiautomatic ... [Read more...]

Today’s post comes from an idea and some starting code by my colleague David Diviny from Nous Group. A common real-world problem is trying to estimate an unknown continuous variable from data that has been published in lumped-together bins. Often th... [Read more...]

Last week I wrote about time-series cross-validation, and mentioned that my original motivation was forecasting unemployment. Actually, I’m interested in nowcasting unemployment rates - that is, estimating a value for a current or recently past perio... [Read more...]

[Read more...]

I usually write blog posts that include big chunks of R code and deal with analysis of specific datasets that I hope are of interest to people with specialist statistical and data science skills (or hoping to develop those skills). But I happen to thin... [Read more...]

Like most people, I was surprised at yesterday’s result for the Australian federal election. But I was much less surprised than most people. The two-party-preferred vote for the ALP (currently 49.11%, with a lot of pre-polling still to count) came do... [Read more...]

So, we’re down to the last few days before the Australian federal election, the first one that I’ve been tracking polls and making forecasts for. I thought I’d address a couple of points raised on Twitter about my forecasts. My forecasts are gene... [Read more...]

The Australian Election Study is an impressive long term research project that has collected the attitudes and behaviours of a sample of individual voters after each Australian federal election since 1987. All the datasets and documentation are freely ... [Read more...]

My eye was caught by this tweet by Robin Wigglesworth of the Financial Times with an Alan Smith animation of the US Treasury yield curve from 2005 to 2009. It’s a nice graphic and it made me wonder what it would look like over a longer period. The ?... [Read more...]

OK, no stats today, just fidgeting about with graphics devices and type faces. Caveat - the details of graphics formats and typefaces is not my area of real expertise, this blog post is me noting down things that I found useful and others might too. As... [Read more...]

The election forecasts Building on my recent blog posts, I’ve put up a page dedicated to forecasts of the coming Australian federal election. It takes the state space model of two-party-preferred vote from my first blog on polls leading up to this e... [Read more...]

Warning - this post discusses gambling odds and even describes me placing small $5 bets, which I can easily afford to lose. In no way should this be interpreted as advice to anyone else to do the same, and I accept absolutely no liability for anyone... [Read more...]

Warning - this post discusses gambling odds and even describes me placing small $5 bets, which I can easily afford to lose. In no way should this be interpreted as advice to anyone else to do the same, and I accept absolutely no liability for anyone... [Read more...]

So now that I live in Melbourne, to blend in with the locals I need to at least vaguely follow the AFL (Australian Football League). For instance, my work like many others has an AFL footy-tipping competition. I was initially going to choose my tips ba... [Read more...]

Swings are far from uniform Last week I introduced a Bayesian state space model of two-party-preferred voting intention for Australian federal elections. It treats the surveys from various polling firms as imperfect (potentially systematically imperfe... [Read more...]