Fuzzy Clustering of VFL/AFL Grand Final Scores

/

Much has already been written about the lamentable and historic-for-all-the-wrong-reasons 2014 Grand Final, which got me to wondering about exactly how atypical it was. Have there been similar Grand Finals and, if so, when?

THE METHOD AND THE DATA

To answer that question we need, of course, a means of determining similarity and to that end I'm going to employ a statistical technique known as clustering, in particular a variant of it known as fuzzy clustering. Clustering techniques generally serve to identify groups of observations within a set that are, in some mathematical sense, more similar to one another than they are to observations in other groups. In the fuzzy variant of the technique - unlike, for example, partitioning around medoids (or PAM) clustering, which I used 5 years ago in another blog on Grand Final Typology - the cluster membership of any observation is permitted to be only partial, or "fuzzy", with each observation potentially belonging to one or all of the identified clusters to varying degrees.

It's not very helpful, however, if most observations wind up being categorised as a mixture of several different cluster types (which is what, for example, archetypal analysis does, though there the archetypal profiles are deliberately selected for their extreme or convex-hull defining nature). An ideal solution will have most observations determined to be entirely or mostly of a single cluster type, with only a few observations having significant levels of membership of more than one or two cluster types.

(There is a wikipedia discussion of fuzzy clustering, and its R implementation is via a function named fanny [which was named in Europe, if you were wondering, but without the benefit of a UK advisor it would seem] which comes in the cluster package.)

So, I have my technique. What about my data?

We could define, I'm sure, literally hundreds of dimensions on which the Grand Finals of different seasons could be compared - crowd size, ladder position of the participants, weather, pre-game favouritism, and so on - but the dimensions I've chosen are all scoring-related as these are both readily available and undeniably salient.

For the analysis then I defined each Grand Final in terms of seven inputs:

  • The winning team's lead at Quarter Time
  • The winning team's lead at Half Time
  • The winning team's lead at Three-Quarter Time
  • The winning team's lead at Full Time
  • The winning team's lead at Half Time compared to Quarter Time
  • The winning team's lead at Three-Quarter Time compared to Half Time
  • The winning team's lead at Full Time compared to Three-Quarter Time

In summary, each Grand Final's signature is a 7-dimensional vector of the winning team's lead at every change and the increase in that lead from one quarter to the next.

Fuzzy clustering, like all clustering algorithms, requires a distance metric to quantify how different one observation is from another, and the fanny implementation offers several. I went with fanny's default metric, which is Euclidean distance (ie the square root of the sum of the squared differences between chosen points on all 7 dimensions).

THE RESULTS

The fanny algorithm provides a number of diagnostic metrics to indicate the quality of a potential cluster solution. One of those is silhouette width, which I used to select a nine cluster solution that seemed reasonably "crisp" - that is, it allocated most Grand Finals to mostly a single cluster but allowed a few to sit mostly across two or three clusters.

Another of fanny's diagnostic outputs is the Dunn Index for a cluster solution, which measures its crispness and, when standardised, lies between 0 and 1 with higher values denoting crisper solutions. As evidence for my qualitative comment above about the apparent crispness of the clustering schema I selected, my nine cluster solution has a standardised Dunn Index of 0.74. Dunn and done then.

The graphic below provides a description of each of the nine cluster types and is roughly ordered by the ease of victory of the winning team with types related to easier victories towards the top and types related to harder-fought victories towards the bottom.

On the left of the table are selected quantile values for each of the input variables for a particular cluster type. So, for example, for Cluster 9, the leads of the winning team at the end of Q1 range from -5 to +23 points with a lower quartile of 3.5 points, a median of 13 points, and an upper quartile of 23 points.

In the middle, the Description column summarises, in words, the key features of the cluster with those features shown in bold being the most defining.

On the right are, firstly, the years of the Grand Finals that are most archetypical of the cluster type. These years are shown in red and any of the years recorded here have been assigned a cluster membership percentage of at least 80% (note that an observation's total membership across all clusters adds to 100% and that no cluster membership can be negative).

Lastly, on the far right are seasons that belong mostly to the cluster in question but which also have significant membership percentages for other cluster types. So, for example, 1957, though it has its highest membership percentage for Cluster 9 at 43%, also has a membership percentage of 29% for Cluster 6.

Games belonging to the first three cluster types are similar in that most saw the winning team lead at every change, though a handful saw the winning team concede small leads at Quarter or at Half Time. Cluster 9 is further characterised by the eventual size of the final margin of victory, with every game in that cluster seeing at least a 10-goal victory. Cluster 8 sees slightly less emphatic victories, in particular with less dominant final terms. This year's Grand Final falls into this category - as, it turns out, did the Grand Final of 2000 between Essendon and Melbourne in which the Dons' leads at each change were 11, 41, 57 and 60 points. The Hawks' on Saturday were 20, 42, 54 and 63 points; the resemblance is obvious and striking.

Cluster 6 has slightly less emphatic wins still and is also characterised by somewhat less dominant final terms.

Cluster 4 contains Grand Finals where the winning teams had mostly built a lead by Three-Quarter Time but then dominated the final term, outscoring their opponents by at least four goals. The Grand Final of 2011 is the most-recent member of this cluster, though it's by no means an archetypical one. That year saw Geelong defeat Collingwood having led by a point at Quarter Time, trailed by 3 points at Half Time, lead by 7 points at Three Quarter Time, and then go on to win by 38 points.

The next cluster, Cluster 3, is another representing Grand Finals where the eventual winners led at every change. In these games, however, such a dominant lead had been established by Half Time - at least 4 goals in every case - that the winning teams could afford to coast for the remainder of the contest either losing or only narrowly winning each of the two remaining Quarters.

Grand Finals from Cluster 5 are characterised by the fact that their eventual winners tended to trail at Half Time before dominating the Third Quarter, mostly by 4 goals or more, and then slightly extending their leads in the final term. About half the winning Grand Finalists in this cluster won by less than 3 goals.

Cluster 1 is another cluster where the Third Quarter proved to be the "Championship Quarter", though here the dominance in that Quarter is generally less than in games from Cluster 5. Winners from seasons in this cluster type, unlike those from Cluster 5, had generally established a modest lead by Half Time. Almost none won the final term of their games and over three quarters of them eventually won by less than 3 goals.

Cluster 2 is similar to Cluster 1 in terms of the profile of leads at Quarter Time, but different in that the winning teams in Cluster 2 Grand Finals had all established a lead by Half Time, wich they maintained until the end of the contest despite mostly losing, mostly narrowly, Third Quarters, and losing or only narrowly winning final terms. Final victory margins in games from Cluster 2 also tend to be slightly smaller than those from Cluster 1. Both the 2012 and 2013 Grand Finals were from Cluster 2.

Finally, Cluster 7 contains Grand Finals where the winning teams almost universally trailed at the end of the first three Quarters before dominating the final term to win, in most cases by only about 2 goals or less. These are the quintessential come-from-behind victories, and it's no surprise that the 1921 Grand Final appears on this list since it was revealed as the archetypical Come-From-Behind victory in the previous blog on the topic of Grand Final typology back in 2009. Coincidentally, the 2009 Grand Final is also categorised here and was a Grand Final that saw the Cats trail the Saints by 2, 6 and 7 points at each of the first three changes, only to outscore the Saints by 19 points in the final term to win by 12 points.

The chart below shows the cluster membership - or the genome, if you like - of every Grand Final and is sorted firstly by a Grand Final's predominant type and then, within type, from the most to the least archetypical.

To finish, let's review the cluster mix of Grand Finals from each of the eras of VFL/AFL football. For the purposes of this table I've ordered the cluster types to match the ordering in the earlier table, which I've loosely interpreted as an ordering by attractiveness, the notion being that harder-fought, come-from-behind victories are more attractive than coast-to-coast blowout victories. As ever, YMMV, especially if the coasting team was your own.

On this analysis, the modern era hasn't been too bad in terms of attractiveness, with over 30% of the Grand Finals falling into Clusters 2 and 7, the most attractive types. The era has been dominated, though, by Grand Finals of Type 5 where three quarters of the victories are by about 4 goals or more - so comfortable if not emphatic. 

Certainly, the trend away from Grand Finals of Types 9 and 8, which dominated the previous era from 1978 to 1995, should be welcomed. Let's hope that this year's Type 8 Grand Final is merely an echo of times past and not a harbinger of things to come.