Are Subscription Sources More Accurate?

Introduction

In this article, we examine whether there are advantages to paying for subscription fantasy football projections. We tested whether projections from subscription sources have higher accuracy than projections from free, publicly available sources. There are arguments that subscription sources would possibly be more accurate as you may expect to get better accuracy as part of what you are paying for. We examined projections from the 2015 season for QB, RB, WR and TE positions.

Overall accuracy

We calculated the projected seasonal points based on standard scoring settings as used in our Projections tool and compared with the actual points. For the aggregation of sources we used the regular mean. We have 10 free sources and 6 subscription sources. The free sources were: CBS, Yahoo, ESPN, FOX, NFL, FFToday, NumberFire, EDS Football, WalterFootball and RTSports. Because the subscription sources are not publicly available we chose not to disclose the names of the sources. For each of the groups we calculated R² (higher is better) and MASE (lower is better) values as well as values for both groups combined. The results are below.

Based on the results, the subscription sources were less accurate than the free sources but add to the overall accuracy. In light of that, one possible reason the subscription sources were less accurate could be because there were more free sources than subscription sources. To investigate that possibility, we examined the accuracy of all possible combinations of 6 sources among the free sources. The results below show the mean R² and MASE values for all the combinations. Reducing the number of free sources did reduce the projections’ accuracy, but the free sources were still more accurate than the subscription sources. In other words, even after accounting for how many sources of projections were included, free projections were still more accurate than subscription projections.

Position Accuracy

Let’s examine whether the results are different when we look at individual positions:

For the QB, WR, and TE positions, the free sources were more accurate than the subscription sources, while the subscription sources were slightly more accurate for the RB position measured by R² but slightly less accurate as measured by MASE. For every position except the QB position, combining the free and subscription sources also increased the overall accuracy.

We also calculated all possible combinations of 6 sources among the free sources by position. As the results below show, the accuracy for the free sources did decrease.  However, as was the case with overall accuracy, the free sources were still more accurate than the subscription sources even after accounting for how many sources of projections were included.

Conclusion

We have seen that subscription sources are not more accurate than the free sources. In general, free projections were actually more accurate than subscription projections.  However, including subscription projections did improve the accuracy of projections both overall and for each position (except quarterbacks). As we have demonstrated before, individual analysts do not reliably beat the “Wisdom of the Crowd” and this analysis further supports that. So while free sources seem to be more accurate, on average, than subscription sources, it is combining them that adds to the accuracy of the overall projections. The most accurate projections combined free and subscriptions sources.  So if you are asking whether you should use free or subscription sources for your projections, the answer is: use both!

You can find data and script for the analysis here.