Evaluation of the UEFA Euro 2020 group stage forecast

A look back on the group stage of the UEFA Euro 2020 to check whether our hybrid machine learning forecasts based were any good…

How surprising was the group stage?

Yesterday the group stage of the UEFA Euro 2020 was concluded with the final matches in Groups E and F so that all pairings for the round of 16 are fixed now. Therefore, today we want to do address two questions regarding our own probabilistic forecast for the UEFA Euro 2020 based on a hybrid machine learning model that we have published prior to the tournament:

1. How good were the predictions for the group stage? Were the actual outcomes surprising?
2. How can we update the forecasts for the knockout stage starting with the round of 16 on the weekend?

The first of these questions is answered in this post while the second question will be deferred to tomorrow’s post.

TL;DR All of our predictions worked quite well and most results were within the expected range of random variation. All tournament favorites proceeded to the round of 16 and mostly the weakest teams dropped out of the tournament. Only in Group E the final ranking was a bit more surprising with Spain ending up second behind Sweden and Poland finishing last and dropping out. At the individual match level there were a couple of games where the clearly stronger team failed to take the win, especially Hungary’s two draws in the “killer group” F were a bit surprising. But other than that the more exciting part of the tournament ist still ahead of us!

Group stage results

First, we look at the results in terms of which teams successfully proceeded from the group stage to the round of 16. The barplot below shows all teams along with their predicted winning probability for the entire tournament, with the color highlighting elimination from the tournament prior to the knockout stage.

Clearly, only teams from the lower half were eliminated with the most unexpected drop-out being Poland. Also, it may seem somewhat surprising that both the Czech Republic and Ukraine “survived” the group stage but with four out of six third-ranked teams advancing to the round of 16 this is not very unexpected.

Looking at the rankings in each group in a bit more detail we see that most group results are as expected. Only in Group E the ranking is really a surprise with Sweden playing stronger than expected and even winning the group. On the other hand, Poland’s performance was somewhat disappointing (as already mentioned above) and Spain waited until the third game (a 5-0 win against Slovakia) to show their full potential.

Match results

After seeing that all the favorites prevailed and only relatively weak teams dropped out of the tournament, we take a closer look at the 36 individual group-stage matches to check whether we had any major surprises. The stacked bar plot below groups all match results into four categories by their expected goal difference for the stronger vs. the weaker team.

In the first bar the stronger team was expected to be only marginally better, with 0 to 0.25 more predicted goals on average. In this bar we see that the stronger team won half of the matches (4 out of 8) while the other half was either lost (3 matches) or ended in a draw (1 match). In short, the distribution of match outcomes conforms essentially exactly with the predictions.

The same is true for the second and third bar where the expected goal difference for the stronger team was between 0.26 and 0.6 or between 0.6 and 1, respectively. The stronger team won in 70.0% out of ten and 77.7% out of nine matches, respectively, thus conforming closely with the predictions.

Only in the last bar with the highest expected goal differences (between 1 and 2 goals), the picture is somewhat unexpected.

1. There were three draws (out of nine matches), two of which by underdog Hungary against the much stronger teams France and Germany. Ultimately, Hungary nevertheless finished last in Group F.
2. One of these nine matches was even lost by the clear favorite but this match was the 0-1 of Denmark vs. Finland. During this match, Danish key player Christian Eriksen suffered a cardiac arrest and had to be reanimated in the stadium before being brought to the hospital. Denmark then had to continue playing the match later that evening and were clearly still under shock. Needless to say that no forecasting model (that we are aware of) would incorporate such extreme and rare events.

As a final evaluation we check whether the observed number of goals per team in each match conforms with the expected distribution based on the Poisson model employed. This is brought out graphically by a so-called hanging rootogram.

The red line shows the square root of the expected frequencies while the “hanging” gray bars represent the square root of the observed frequencies. This shows that the predictions conform closely with the actual observations. There were only a few more occurrences of three goals (ten times) than expected (6.1 times) but this deviation is also within the bounds of random variation.