TSstudio 0.1.2

Backtesting

The ts_backtesting function provides you the ability to train, test and evaluate multiple models with the use of backtesting approach. This allows automating the forecasting process by running a ‘horse race’ between different models or approaches while testing them over multiple periods to evaluate their performance over time. The example below demonstrated the use of the function to forecast the monthly consumption of natural gas in the US for the next five years (or 60 months). By default, the function is testing seven different models, using expended window over six periods:

library(TSstudio)
data("USgas")

ts_info(USgas)
##  The USgas series is a ts object with 1 variable and 225 observations
##  Frequency: 12 
##  Start time: 2000 1 
##  End time: 2018 9
usgas_backtesting <- ts_backtesting(USgas,
                                 periods = 6, # Set the number of periods in the backtesting
                                 window_size = 12, # Set the length of the testing set
                                 h = 60, # Set the horizon of the final forecast
                                 plot = FALSE,
                                 error = "MAPE" # Set the error matrix
                                 )
##    Model_Name  avgMAPE    sdMAPE  avgRMSE   sdRMSE
## 1  auto.arima 6.148333 0.9886843 184.1150 15.82619
## 2      hybrid 6.691667 0.6159681 196.2350 16.66238
## 3      nnetar 7.195000 1.1111571 255.5100 23.01805
## 4       tbats 7.468333 0.7641313 208.5633 17.60290
## 5        bsts 7.581667 1.8553966 215.4200 44.64997
## 6 HoltWinters 7.736667 0.6686903 218.5017 19.05705
## 7         ets 7.778333 1.7040824 202.5533 45.95689

By default, the model which performed the best in the testing sets, according to the error criteria (RMSE or MAPE), will be selected by the function to forecast the series. In the case of the USgas series, as you can see in the leaderboard above, the auto.arima model achieved the best results and therefore, the function will select this model to forecast the future values:

# Plotting the results
usgas_backtesting$summary_plot

In addition, the output of this function includes all the output from the trained models and their forecasts. For instance, you can check the residuals of the selected model:

check_res(usgas_backtesting$Models_Final$auto.arima)

Or pull the forecast of the hybrid model:

plot_forecast(usgas_backtesting$Forecast_Final$hybrid)

A short video of this function is available here

The ts_seasonal function

The ts_seasonal function is now supported data frame objects (data.frame, data.table, and tbl), in addition to the time series objects (ts, xts, and zoo). The function has three modes, which can define with the type argument:

• normal – subsetting and plotting the series by its full cycle (or year), this allows identifying if there is a repeated pattern in the series from year to year

ts_seasonal(USgas, type = "normal")