Global energy forecasting competitions

The 2012 GEFcom competition was a great success with several new innovative forecasting methods introduced. These have been published in the IJF as follows:

• Hong, Pinson and Fan. Global Energy Forecasting Competition 2012
• Charleton and Singleton. A refined parametric model for short term load forecasting
• Lloyd. GEFCom2012 hierarchical load forecasting: Gradient boosting machines and Gaussian processes
• Nedelec, Cugliari and Goude: GEFCom2012: Electric load forecasting and backcasting with semi-parametric models
• Ben Taieb and Hyndman. A gradient boosting approach to the Kaggle load forecasting competition
• Silva: A feature engineering approach to wind power forecasting: GEFCom 2012
• Mangalova and Agafonov: Wind power forecasting using the k-nearest neighbors algorithm

These papers will appear in print in the next issue of the journal. The data and code for each method is also available.

GEFCom 2014

The 2014 edition of the competition is now under way with a probabilistic focus. The great thing about these competitions and the associated papers is that they provide a serious empirical comparison of the forecast capability of different algorithms. There are lots of suggested approaches to forecasting energy, but without head-to-head comparisons like this, it is not always clear what is a good method, and what methods can be discarded.

The developers of the best performing methods in the competition will be invited to submit a paper to a special issue of the IJF on probabilistic energy forecasting, to be edited by Tao Hong and Pierre Pinson.

In addition to the regular competition, there is also a competition for students where the best-performing classes can win a prize. See Tao Hong’s blog for further information.

Evaluating forecast distributions

Because this round of the competition is focusing on predicting the whole distribution, and not just a point forecast, we need a method of evaluating the forecasts which takes account of the whole distribution. We have chosen to use quantile scoring.

For each time period throughout the forecast horizon, the participants should provide the quantiles — call these , with or the natural lower bound, and or the natural upper bound. The full predictive densities composed by these quantile forecasts are to be evaluated by the quantile score calculated through the pinball loss function.

For a quantile forecast with as the target quantile, this score is defined as:

where is the observation used for verification, and .

To evaluate the full predictive densities, this score is then averaged over all target quantiles, from 0.01 to 0.99, for all time periods over all forecast horizons. The lower the score, the better the forecasts are.

Further information

For more information about the competition and how to enter, please visit www.gefcom.org.