Comparing predictions: World Cup scores

As many others too, me and some colleges at STATWORX took part in a little betting game for the World Cup 2018. Since the group stage is over, I was wondering how well – or better – how worse my prediction was. I am comparing my result with other predictions by using the point system of the betting game. All functions, code and data can be found at our Github page.

Which prediction methods to use?

Since I don't follow soccer at all but I like rolling dice, I thought to just randomly predict the result for each game. The follow up question was: How? Do I just predict the number of goals for each team and assume a goal's distribution? Do I use additional information like betting odds or FIFA ranking? Do I just lay back and watch some games?

At the end I decided to predict the result as a whole and not for each team. Since I didn't want to assume a distribution I used all past World Cups results and took a sample of 48 (the number of games in the group stage this time).

During my search for other prediction results I came across this blog post on R-bloggers which was build on a talk by Claus Thorn Ekstrøm at the eRum in May 2018. His ideas can be found here and were the foundation of the fuctions I used in my prediction.
I also found this nice World Cup simulator from a team at the Goethe-Universität Frankfurt. Here you can simulate the whole World Cup with all results.

So I had my own and some other methods for the prediction – all I was missing was data!

How to get the data?

Historical data

On the official FIFA website I found tables for all past World Cup result, which I could just copy into some textfiles and than search them for the results. I distinguished between a 0-1 and a 1-0 since there is an order in the betting game. The distribution of all historical group stage games looked like this:

Scrapping the online simulation

Here I would like to say thanks to my colleague Moritz, who worte a python script for me, that clicked the simulation button 100 times and copied it into an csv file. Also thanks to the guys from fussballmathe for providing the online simulation tool.

To compare the simulation scores with the other ones, we had to make a few adjustments:

• the data had to be translated from german into englisch
• the order of the games needed to be the same as the others (just as a convenience)
• some of the scores were switched, so we transformed them form A - B to B - A

Simulating the other methods

As mentioned above, the main functions originated from Claus Thorn Ekstrøm. The play_game function he wrote uses three different method to predict the scores:

1. Using two independant Poisson distributions, with no further information about the teams.
2. Using a Skellam distribution (difference of Poissons) when there is a restriction on the parameters based on the skill levels of the teams.
3. Using the ELO ratings as additional information to predict the scores.

Luckily, Ekstrøm provided data for these functions on his github page, so that I could just use them for the comparison. Furthermore, I added the historical and the other simulation to the play_game function.

For every method I simulated 100 World Cups to get an overview of which method may produce the best outcome. Now I had all the data and just needed to apply the point system of the betting game to compare the results.

How does the point system work?

To compare all the different prediction and evaluate them I used the point system of the betting game which got me into this problem at the first place. The system is quite simple, you get points depending on how good your bet was. If your bet was spot on, you get four points. If your goal difference was right, you still get three. If you at least had the same winner or also a draw you get two points. For everything else – well zero points!

Who did it the best?

So all preparations were done and I just had to wait for the real scores to come in!

We can see, that there are some methods, which seems to have the same outcome:

• the historical data and the Poisson distribution
• the Skellam distribution and fussballmathe's simulation

The best method to predict the result seems to be the one based on the ELO ratings. This is not so surprising, since it used the most relevant information.

The last graph shows the result of all participants in the betting game. One could say, that expert knowledge is better than chance – but not always. ?
The red line represents the mean of the method and the two black lines show my bet and the points I got … well nearly. When I had to make the bet, I accidentally did not load the last three World Cups for the historical data. That's why my actual points are 34 and not 38. So yes, I came in dead last – but with honor and my head held hight!

But whatever result we were predicting – let's enjoy the final round of this year’s World Cup!

Über den Autor

Jakob Gepp

Jakob ist im Statistik Team und interessiert sich im Moment stark für Hadoop und Big Data. In seiner Freizeit bastelt er gerne an alten Elektrogeräten und spielt Hockey.

Der Beitrag Comparing predictions: World Cup scores erschien zuerst auf STATWORX.