Comparing the Men's AFL Score-by-Score Simulator to Historical Results

/

In the last blog, I described the creation of a simulator that could be used to generate synthetic games meant to resemble actual games from the men’s AFL competition.

Based on the manner of its construction, I firmly asserted that “It produces scores with a broadly realistic event-to-event cadence both in terms of the time between events and the likely sequence of them given what’s gone before in the same game”, but didn’t provide much evidence to support that contention.

Today I want to provide some supporting evidence for the defence.

SIMULATING GAMES FROM THE LAST DECADE

In that previous blog, I used home-and-away games from 2010 to 2019 to create the next event and time to next event models, but then only simulated games from the 2017 to 2019 seasons.

Today I’m going to simulate games from the entire 2010 to 2019 period, for which purpose I’ve made two tweaks to the simulation code.

  1. Previously, I provided a trial-and-error motivation for the covariance matrices for expected scoring shots and conversion rates. I went back and analysed the differences between MoSS2020 pre-game scoring shot forecasts and actual scoring shot production over the 2010 to 2019 period, and found that the errors had a standard deviation of 5.4 for home teams, 5.2 for away teams, and a correlation of -0.295. I’ve altered the covariance matrix for injecting scoring shot variability accordingly.
    Similarly, I analysed the differences between actual scoring shot conversion and the assumption of fixed rates (52.8% for home teams, and 52.9% for away teams) and found that the errors had a standard deviation of 11% points for home teams and away teams, and a covariance of -0.02. I’ve altered the covariance matrix for injecting conversion rate variability accordingly.

  2. Whereas for the previous blog we multiplied estimated home team score as next event probabilities by 0.9895, here we multiply home team next event probabilities by 0.994 and multiply away team next event probabilities by 1.0132.
    (If the multiplication of the away team probabilities makes the sum of the home team and away team probabilities greater than 1, we set the away team probability to 1 minus the home team probability, and the probability for a quarter end equal to zero.)

We then use the amended simulation code to create 20,000 simulated games from the 2010 to 2019 period, starting with the actual scoring shot production data for games from that period, selecting a game at random, and then adding the on-the-day effects as before.

Again we get very close agreement across the replicates between the actual and the expected home team scoring, away team scoring, and game margins.

CLUSTERING THE SIMULATED GAMES BY WINNING TEAM MARGIN TRAJECTORY

In another, earlier blog, we clustered the home-and-away games from the 2010 to 2019 era based on the trajectory of the winning team’s progressive margin across the game. One test of how realistic are the simulator outputs would be to cluster them on the same basis. If the clusters that emerge from the simulated data are similar to those from the real data, that would provide some empirical evidence for my previous assertion about the real-life mimicking qualities of the simulator.

If we do that on a random sample of 10,000 of the 20,000 simulated games (the clustering algorithm scales poorly with sample size, so using the entire 20,000 synthetic games simply wasn’t feasible) the following optimal clustering solution emerges.

(Recall that the grey lines are each a single game, and the red lines track the median winning team lead across all the games from the relevant cluster, measured at 5% point game fraction increments.)

Some of those 18 clusters look quite similar to those from the 20 cluster solution we found for the real data, which I’ve reprised below.

We can make the comparison between the two more direct by plotting, for each of the clusters in the solution for the simulated data, the nearest one or two clusters from the solution for real data, where proximity is measured by the sum of the absolute distances between the relevant median tracks for the clusters (ie the red lines).

Most of the clusters in the solution for the simulated data have close analogues in the solution for the real data, though a couple of simulated clusters (viz 3 and 13) might best be described as a mixture of two real data clusters. Simulated clusters 7 and 13 both share some similarities with real data cluster 11, and no simulated cluster is closest to real cluster 3, although it is simulated cluster 14’s second nearest-neighbour.

CONCLUSION

The code to simulate men’s AFL games does seem to produce game score progressions that are broadly realistic.