Time series cross-validation 2

In my previous post, I shared a function for parallel time-series cross-validation, based on Rob Hyndman’s code.  I thought I’d expand on that example a little bit, and share some additional wrapper functions I wrote to test other forecasting algorithms.  Before you try this at home, be sure to load the cv.ts and tsSummary functions from my last post.



These functions add random walk models, the theta method, structural time series, seasonal decomposition,  and simple mean forecasts to our cross-validation repertoire.  The following code fits each of these models to the example dataset, and charts their accuracies out to a forecast horizon of 12 months.  Note that none of this code runs in parallel, but if you wish, you can parallelize things by loading your favorite foreach backend.  I would suggest running meanf, rwf, thetaf, and the linear model before loading a parallel backend, as all of these methods run very fast and do not need parallelization.


Here is the resulting figure.  As you can see, the mean forecast is very inaccurate, but provides a useful baseline.  The random walk forecast and the theta forecast are both improvements, but they ignore the function’s seasonal component and have a clear seasonal error pattern.  StructTS and STL are clustered down at the bottom with accuracies similar to the linear model, arima model, and exponential smoothing model: