[This article was first published on Fantasy Football Analytics in R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
In prior posts, I demonstrated how to download, calculate, and compare fantasy football projections from ESPN, CBS, and NFL.com. In my last post, I demonstrated how to download FantasyPros projections, which aggregate projections from many different sources to increase prediction accuracy. In this post, I will compare fantasy football projections from ESPN, CBS, NFL, and FantasyPros, including our average and latent projections to determine who has the best fantasy football projections.Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
The R Script
The R Script for downloading fantasy football projections from FantasyPros is located at:https://github.com/dadrivr/FantasyFootballAnalyticsR/blob/master/R%20Scripts/Evaluate%20Projections.R
To compare the accuracy of the projections, we will use various metrics including:
- R-squared (R2) – higher is better
- Harrell’s c – higher is better
- Somer’s Dxy – higher is better
- Intraclass correlation (ICC) – higher is better
- Mean absolute error (MAE) – lower is better
- Root mean squared error (RMSE) – lower is better
- Mean absolute percentage error (MAPE) – lower is better
- Mean absolute scaled error (MASE) – lower is better
Whose Predictions Were the Best?
The results are in the table below. The rows represent the different sources of predictions (e.g., ESPN, CBS) and the columns represent the different measures of accuracy. The “average” variable represents the mean of projections from ESPN, CBS, NFL.com, and FantasyPros. The “latent” variable represents the common variance among projections from ESPN, CBS, NFL.com, and FantasyPros. The source with the best measure for each metric is in bold.
| Source | R-squared | Harrell’s c | Somers’ Dxy | ICC | MAE | RMSE | MAPE | MASE |
|---|---|---|---|---|---|---|---|---|
| ESPN | .497 | .725 | .450 | .695 | 44.18 | 56.23 | 43.66 | .596 |
| CBS | .607 | .775 | .549 | .754 | 41.37 | 53.63 | 59.07 | .518 |
| NFL.com | .487 | .743 | .486 | .655 | 48.80 | 62.47 | 38.39 | .701 |
| FantasyPros | .667 | .775 | .549 | .816 | 32.66 | 45.36 | .434 | |
| Average | .657 | .776 | .551 | .810 | 33.62 | 46.18 | .447 | |
| Latent | .661 | .779 | .559 | .810 | 34.31 | 46.96 | 76.38 | .441 |
Note: MAPE was unable to be calculated for FantasyPros and the average because of values of zero in the series (for a discussion on this topic and for reasons to prefer MASE to the other error metrics, see here).
Here is how the projections ranked when focusing on R-squared and MASE:
- FantasyPros
- Latent
- Average
- CBS
- ESPN
- NFL.com
In general, projections from FantasyPros were more accurate than projections from ESPN, CBS, and NFL.com, and were also more accurate than our average and latent variables. FantasyPros projections explained about 67% of the variance in the actual points scored in my Yahoo league in the 2012 season. Interestingly, the average of the sources was more accurate than any of the individual sources. Even better than the average was a latent variable representing the common variance of the sources, which discards the unique, error variance.
Here is a scatterplot of the FantasyPros projections in relation to the actual points scored:
Conclusion
The best fantasy football projections in 2012 were from FantasyPros, a site that averages across numerous sources of projections. FantasyPros projections explained 67% of the variance in players’ actual points scored in 2012, and were more accurate than projections from ESPN, CBS, and NFL.com. Now you know where to turn to get the best projections for your fantasy football league (note that FantasyPros have not yet updated their projections for 2013).
To leave a comment for the author, please follow the link and comment on their blog: Fantasy Football Analytics in R.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.