Explaining Variability in Game Margins

Some seasons are notable for the large number of blowout victories they force us to endure - a few recent seasons come immediately to mind - while others are more memorable because of their highly competitive nature. To what extent, I've often wondered, could we attribute a season full of sizable victory margins to the fact that strong teams were more often facing weak teams, making the magnitude of the defeats predictable if still lamentable, versus instead attributing them to on-the-day or random events that were genuinely unforeseeable pre-game?

In a similar vein, I blogged recently about the 10 things I've found most surprising in my analysis and modelling of AFL so far and, in the final entry on that list, expressed surprise about the relatively small proportion of the variability in home team game margins - the difference between the points scored bythe home and the away team - that could be explained using only the TAB Bookmaker's pre-game prices. That proportion, measured over the seasons 2006 to 2013, the period for which I have trustworthy Bookmaker data, was about 35%.

In this blog then I'll be estimating the relative contributions of differences in team abilities versus differences in luck to the final outcome of contests. Specifically I'll be analysing how the proportion of explainable variability in game margins has changed from one season to the next and from the regular home-and-away portions of seasons to the Finals, and investigating the extent to which this variability might instead be attributed to on-the-day or truly random factors, as I mentioned earlier.

THE APPROACH

You need to grant me two assumptions to derive the results in this blog:

• That we can think about the home team game margin for any particular game as being the sum of two random variables, the first attributable to known, pre-game factors such as the relative abilities of the teams, their recent form and the venue for the game, and the second attributable to on-the-day, random and other factors that weren't known pre-game.
• That the TAB Bookmaker's head-to-head prices, when converted to an implicit prediction of the game margin is an excellent proxy for the first component - that is, of the expected value of the portion of the game margin attributable to everything that was knowable pre-game.
  (I could, of course, instead use the points start from the handicap market as an estimate of the Bookmaker's expected game margin, but his treatment of games where the start has been, in absolute terms, 6.5 points or less has differed over the years, which makes it difficult to come up with reliable estimates for these games.)

View fullsize

Granted those assumptions and given data for a number of games we're able to calculate:

• The variability of actual game margins, and of the explainable and unexplainable portions of those margins
• The covariance (and correlation) between actual game margins and the explainable component
• The proportion of the variability in actual game margins that can be explained by known, pre-game factors 

The mathematical and statistical details of how we can derive these numbers appear in the grey sidebar at left, but they follow logically from the two assumptions listed above.

Of the two assumptions, the second is the more important and is pivotal in the process of allocating the observed variability in game margins to the two putative causes. If it is instead the case that the TAB Bookmaker's head-to-head prices at least sometimes ignore information that would be relevant in predicting game outcomes - and oh how I wish that were true and I knew what he was missing - then the methodology I've described will overstate the contribution of on-the-day and random factors to the variability in game margins.

I'd assess the likelihood of this being the case as small, certainly if we asserted that it were true systematically, but it is possible that the Bookmaker might occasionally miss something significant (or, more likely, that punters do and the Bookmaker's pricing exploits this oversight).

There's no obvious way to convert a Bookmaker's head-to-head prices into a margin assumption but, that aside, there aren't many moving parts to the analysis, so the results should be robust.

What we find empirically, as described in the sidebar, if we look at the entirety of the period 2006 to 2009 is that:

• Game margins across that span of history had a standard deviation of a little over 45 points (around, though it's not shown there, a mean of about 7.5 points per game).

• The standard deviation of the TAB Bookmaker's implicit margin predictions for those same games was 25.6 points (around a mean, though not shown here either, of about 8.2 points per game)

• Perhaps most importantly, the standard deviation of the "random" component of game margins was about 36.7 points (about a mean equal to the difference in the means just provided, which is about 0.7 points per game)

• The correlation between the actual and Bookmaker-predicted game margins is +0.59 and the proportion of variability in game margins explained by variability in predicted margins is about 35%.

SEASON-BY-SEASON AND HOME-AND-AWAY VERSUS FINALS ANALYSIS

We can apply the methodology described in the sidebar above to any portion of games from history and determine the values of the same variability and correlational metrics for those games.

View fullsize

Looking firstly at the season-by-season data we see that game margins have been at their most variable over the past three seasons, with the standard deviation tracking in the 8 to 8-and-a-half goal range. The TAB Bookmaker's pre-game head-to-head prices have demonstrated some foreknowledge of this outcome, the standard deviation of his margin predictions rising from the 3 to 4 goal range it had maintained in earlier seasons to about a 4-and-a-half to 5-and-a-half goal range.

The random component of game margins in these same three years was, however, within about one half-goal of the all-season average.

Overall this led to a sharp increase in the proportion of game margin variability explained by the TAB Bookmaker's pre-game prices, lifting it close to 50% in the past two seasons from a low around the 20% mark about half a decade earlier. (This is another way of presenting the case for the relative predictability of recent seasons, which I wrote about in this blog.) 

Comparing the results across all seasons for regular home-and-away season games to results for Finals we find that, as we'd expect since Finals tend to pit more evenly-matched teams against one another, the variability in game margins and in the TAB Bookmaker's predicted margins is much smaller for Finals than it is for home-and-away season games.

The random component, however, is only marginally smaller, so the proportion of game margin variability is substantially smaller for Finals than for home-and-away fixtures.

(I note in passing that variability in game margins is quite different from variability in game outcome - it could be that the teams expected to win did exactly that, just by too much or too little.

The results of Finals are, I'd suggest, much more predictable than the results for home-and-away games even if, as the result above suggests, game margins in Finals are less predictable. If I can come up with a methodology equivalent to that I've used here for analysing game margins but instead for analysing game outcomes I'll do a future blog on this topic.)