At the conference of the New Zealand Association of Economists (NZAE) in late June 2016 I gave a paper on Modelled Territorial Authority Gross Domestic Product, a new dataset my team developed last year in my day job. See the official website of the M... [Read more...]

In my most recent blog post I introduced the dualplot() R function, which allows you to create time series plots with two different scales on the vertical axes in a way that minimises the potential problems of misinterpretation. See that earlier post ... [Read more...]

Are they really as bad as all that? I’ve been mulling over time series charts with two different y axes, which are widely deprecated in the world of people who see ourselves as professional data workers. Looking down on dual axis time series charts ... [Read more...]

Deal with feature selection and collinearity Recently I’ve been making more use of elastic net regularization as a way of fitting linear models to data when I have more candidate explanatory variables than I know what to do with and some of them are... [Read more...]

Introduction A few months back the first, pre-CRAN versions of my nzelect package included some data from the New Zealand Census 2013. As noted in my last post, I’ve now split this into a separate nzcensus package, for ease of development and maint... [Read more...]

Introduction The nzelect R package which I first introduced in a blog post in April is now available on CRAN. The version number is 0.2.0. The difference from version 0.1.0 is sizeable - all the 2013 census data has been removed and is now in a companion package, nzcensus. This is for ease ... [Read more...]

In my last post I had a first look (for me) at Estimated Household Income Inequality data from the University of Texas Inequality Project. These data came to my attention when Professor James K. Galbraith used them in his keynote presentation to the 2016 New Zealand Association of Economists conference. Some ... [Read more...]

I’m at the New Zealand Association of Economists annual conference in Auckland. The opening keynote speech was from James K. Galbraith on a global view of inequality. He showed a variety of results from the University of Texas Inequality Project’s Estimated Household Income Inequality dataset, which I hadn’... [Read more...]

A big 18 month project at work culminated today in the release of new Monthly Regional Tourism Estimates for New Zealand. Great work by the team in an area where we’ve pioneered the way, using administrative data from electronic transactions to supplement traditional sources in producing official statistics. Here’s ... [Read more...]

Last week I gave a seminar for around 40 analysts from another government agency on using graphics to represent data. In doing such presentations, I usually focus on different purposes of graphics: exploratory as part of the analysis workflow (eg as diagnosis for statistical models) for presenting results Exactly what the ... [Read more...]

Modelling strategies I’ve been re-reading Frank Harrell’s Regression Modelling Strategies, a must read for anyone who ever fits a regression model, although be prepared - depending on your background, you might get 30 pages in and suddenly become convinced you’ve been doing nearly everything wrong before, which can ... [Read more...]

Overview How big a sample size would you think you need to get a reliable 95% confidence interval (ie one that really does contain the true value 95% of the time) for a single univariate statistic like standard deviation? 30? 50? Turns out the answer is more like 20,000 for some not-particularly-extreme real-world data. In ... [Read more...]

Robust regression For training purposes, I was looking for a way to illustrate some of the different properties of two different robust estimation methods for linear regression models. The two methods I’m looking at are: least trimmed squares, implemented as the default option in lqs() a Huber M-estimator, implemented ... [Read more...]

Minimalist axes text and tick marks One of the ideas in Edward Tufte’s The Visual Display of Quantitative Information was to minimise non-data-ink by dropping the regular text labelling values on axis guides, and instead using the axis guides to mark the values of the actual data. For those ... [Read more...]

Background and motivation In an earlier post I explored ways that might improve on standard methods for prediction intervals from univariate time series forecasting. One of the tools I used was a convenience function to combine forecasts from Rob Hyndman’s ets and auto.arima functions. David Shaub (with a ... [Read more...]

Motivation This post is the fourth in a series that make up my entry in Ari Lamstein’s R Election Analysis Contest. Earlier posts introduced the nzelect R package, basic usage, how it was built, and an exploratory Shiny web application. Today I follow up on discussion in the StatsChat ... [Read more...]

Motivation This post is the third in a series that make up my entry in Ari Lamstein’s R Election Analysis Contest. First I introduced the nzelect R package from a user perspective. Second was a piece on how the build of that package works. Today, the third in the ... [Read more...]

Motivation This post is the second in a series that make up my entry in Ari Lamstein’s R Election Analysis Contest, Yesterday I introduced the nzelect R package from a user perspective. Today I’m writing about how the build of that package works. This might be of interest ... [Read more...]

The contest Inspired by Ari Lamstein’s R Election Analysis Contest, I’ve fast-tracked a project that’s been at the back of my mind for a while, to make available in a friendly, tidy R package a range of data about New Zealand elections. My entry for the contest ... [Read more...]

The ggseas package for R, which provides convenient treatment of seasonal time series in the ggplot2 universe, was first released by me in February 2016 and since then has been enhanced several ways. The latest version, 0.4.0, is now on CRAN. The improvements since I last blogged about ggseas include: added the ... [Read more...]