PREDICTION ERRORS
This page includes charts relating to the analysis of prediction errors from any of the MoS statistical models. These errors might relate to predictions of:
- Team Scores
- Margins
- Total Scores
ABSOLUTE GAME MARGIN ERRORS AND EXPECTED TEAM SCORES
MoSHBODS MODEL
The heteroskedasticity or otherwise of game margins has been a common topic here on MoS (see, for example, this post from 2014 or this post from 2013). Generally, the evidence for heteroskedasticity has been found to be fairly weak, but such analyses and conclusions depend on:
- the model used to provide forecasts
- the variables analysed as being potentially associated with the residuals of that model (ie the differences between the actual and expected margins
In this first analysis, the MoSHBODS Team Rating System has been used to provide the expected margin data and expected scores for the home and away teams in each contest, these latter variables the ones postulated as being potentially associated with margin prediction errors.
Because average scores and margins have varied over time, we need to perform this analysis on an era-by-era basis. The results appear below.
What is new about this analysis is the technique employed to define and colour each point in the chart. Every point is based on a "neighbourhood", and there are as many neighbourhoods as there are games in the data. A neighbourhood comprises the 50 points nearest to the original point, with distance based on the pre-game expected Home and Away team scores.
For each neighbourhood we calculate the weighted mean of the expected home and the expected away team scores, and a weighted mean of the absolute margin error in those games. The weights we use are the inverse of the distances from the original point that defined the neighbourhood, these distances also measured in terms of expected home team and away team scores. We plot each point at the weighted mean expected scores, and colour it based on the weighted mean absolute margin error.
This technique helps to reveal the underlying spatial patterns in the data by colouring points based on a range of similar games rather than on a single game.
In this chart, the pattern of heteroskedasticity is markedly different in each era.
For the 1897 to 1919 era we see larger average absolute margin errors in games predicted to be mismatches - where the Home team is expected to score 60 or more and the Away team 50 or less, or, alternatively (and to a lesser extent), where the Away team is expected to score 60 or more and the Home team 50 or less.
The pattern for the 1920 to 1939 era is somewhat similar for large expected Home team margins, but harder to generalise for the remainder of the chart. For 1940 to 1959 there's no real pattern at all, while for 1960 to 1979 the largest average absolute margin errors come in the games expected to be high scoring for the Home and the Away teams.
There is, again, no real pattern for the 1980 to 1999 era, and increased average absolute margin errors in the 2000 to 2017 era mostly for games where the Away team is expected to score highly.
Overall, it's hard to postulate any enduring relationship between the expected Home and/or Away scores and the eventual absolute margin errors.
In interpreting these charts it's important to remember that any finding of heteroskedasticity is particular to the model used to define expected margins and is only with respect to the variables analysed. It might well be, for example, that the MoSHBODS model has some feature or flaw that causes it to produce margin estimates that are or are not heteroskedastic either generally or for particular eras - we can't assert that V/AFL game margins are, inherently, heteroskedastic based on any such analysis.
The historical accuracy of MoSHBODS' predictions relative to the TAB Bookmaker suggests that, at least for the modern era, its opinions are similar to those of a motivated expert and so they should not be completely dismissed, but it wouldn't hurt to check another MoS team score predicting model ...
MoSSBODS MODEL
Now we use the MoSSBODS Team Rating System instead to provide the expected margin data and expected scores for the home and away teams in each contest.
The results are broadly similar.
(NB The correlation between MoSHBODS' and MoSSBODS' Home Team score forecasts is +0.695, and between their Away Team score forecasts +0.689.)
ABSOLUTE GAME MARGIN ERRORS AND EXPECTED MARGINS
We might, instead postulate that absolute margin error tends to increase with the size of the expected margin.
MoSHBODS Model
We find no such relationship for MoSHBODS.
MoSSBODS Model
We also find no such relationship for MoSSBODS.
(NB The correlation between MoSHBODS' and MoSSBODS' Home Team Margin forecasts is +0.971.)
ABSOLUTE GAME MARGIN ERRORS AND EXPECTED TOTAL SCORES
Finally, we might postulate that absolute margin error tends to increase with the expected total score.
MoSHBODS Model
At best, there is a very weak relationship between average absolute margin error and expected total score for MoSHBODS for some eras.
MoSSBODS Model
And, similarly, at best, there is a very weak relationship between average absolute margin error and expected total score for MoSSBODS for some eras.
(NB The correlation between MoSHBODS' and MoSSBODS' total score forecasts is about +0.997, so reaching the same conclusion here is the least surprising of all.)
As a final comparison, if we use adjusted Pinnacle opening lines and totals (using the data from here) we obtain yet another view, albeit on not a lot of data.
Absolute Margin Errors vs Expected Team Scores
The pattern in the average absolute margin errors we see here is different from that for MoSHBODS and
Absolute Margin Errors vs Expected Margin
Here too we see no strong relationship across the range of most-common forecasts.
Absolute Margin Errors vs Expected Total
And, finally, we also see no strong relationship between absolute margin errors and expected totals.
