A Pointless Analysis

Lately, my wife and I have been watching a lot of the BBC quiz show, Pointless, hosted by Alexander Armstrong and Richard Osman, in which contestants progress by providing answers that are correct but not well-known by people to whom the same questions were put before the show.

GAME MECHANICS

The quintessential "good" answer to a question in Pointless is one that is "pointless" - that is, both correct but not provided by anyone amongst the 100 people who were given 100 seconds to come up with their own answers. In the first three rounds (the first two of which comprise two "passes") of the show, during which an initial set of four pairs is whittled down to just one, pointless answers add £250 to an ongoing jackpot. That jackpot resets to £1,000 for the next show after it is won, and increments by £1,000 from one show to the next when it is not won. 

In the first two rounds (four passes), pairs work independently to provide one answer each, and the scores are summed within pairs. The pair with the highest sum is eliminated. In the third round, when just two pairs remain, the pairs can collaborate and the winning pair is the one which wins two out of three points by providing the lower-scoring answers.

The final round has the sole surviving pair conferring for 60 seconds in an attempt to find a pointless answer within any of three specified, usually quite narrowly defined, questions. If they find such a pointless answer, they take the jackpot. If they do not, they win a "coveted Pointless trophy" but no money, and do not return for the next show.

Pairs can appear on a maximum of two shows, and will only fail to do this if they are successful in reaching the final pass in their first appearance.

INTERESTING QUESTIONS

As I think I've said before here on the MoS site, analysts are driven by curiosity. We strive to answer questions that, sometimes for no objectively good reason, we genuinely need to answer once we've thought of them.

In the case of Pointless, I got to wondering about the average number of returning pairs from one show to the next. Absent any analysis, I'd have guessed it was about two, maybe a little less.

Working through the game mechanics, it's not hard to convince yourself that the number of returning pairs can range anywhere from zero to three, with four returning pairs being an impossibility because the pair competing for the jackpot in one show cannot return for the next. Zero returning pairs, though logically feasible, seems empirically rare, as does three returning pairs (though of course this must have been the case for, at least, the second game in Pointless history under the four-team format).

A more definitive answer to the average number of returning pairs can be obtained through simulation based on the game mechanics as already described. For that simulation we'll also need to make assumptions about:

• the relative skills of competing pairs (for simplicity, I've assumed that all pairs are equally likely to progress to the jackpot round)
• the probability of providing a pointless answer (we'll run different simulations varying this probability from 2.5% per answer to 10% per answer. Observationally, 5% seems to be about the actual probability)
• the probability of any pair winning the jackpot (we'll also run different simulations varying this probability, though here from 5% to 50% as it seems more difficult to specify a range for this simulation parameter based on observations)

As well as answering our original question about the average number of returning pairs, the simulations will also allow us to investigate how the average amount won per show and average jackpot size vary as we change the two input probabilities.

THE RESULTS

With four different values for the probability of providing a pointless answer, and 10 for the probability of the jackpot being claimed, we've forty different scenarios to simulate. We'll simulate each of them for 1,000 shows.

Let's firstly look at how average winnings per show varies with the probability of providing a pointless answer.

(Please click on this and any subsequent image to access a larger version).

View fullsize

We see that the average winnings per show increases by about £125 for every 2.5% point increase in the probability of providing a pointless answer. In each show (ignoring the possibility of tie-breaking extra guesses) there are either 18 or 20 opportunities for a pointless answer to be provided in the first three passes - eight opportunities in the first pass, six in the second pass, and either four or six in the third pass depending on whether a third question is required.

Assuming that all opportunities are independent (which is almost certainly not true, because some questions have no or few pointless answers, and answers cannot be repeated), the number of pointless answers in a show can be thought of as a Binomial random variable with two parameters:

• mean equal to the probability of providing a pointless answer, and
• number of trials as either 18 or 20.

To model the number of pointless answers in each simulation replicate I randomly chose 18 or 20 for the number of trials, assigning a 50% probability to each (assuming that a third question will be required half the time in the head-to-head pass).

Under these assumptions we can calculate:

• Expected number of pointless answers if probability is 2.5% = 0.5*mean(Binomial(2.5%, 18)) + 0.5*mean(Binomial(2.5%, 20)) = 0.48
• Expected number of pointless answers if probability is 5% = 0.5*mean(Binomial(5%, 18)) + 0.5*mean(Binomial(5%, 20)) = 0.95
• Expected number of pointless answers if probability is 7.5% = 0.5*mean(Binomial(7.5%, 18)) + 0.5*mean(Binomial(7.5%, 20)) = 1.43
• Expected number of pointless answers if probability is 10% = 0.5*mean(Binomial(10%, 18)) + 0.5*mean(Binomial(10%, 20)) = 1.90

On average then, we add about 0.5 more pointless answers every time we increase the probability of a pointless answer by 2.5% points. Since each pointless answer is worth £250, we'd expect that to translate into a £125 increase in average winnings, which is what we observe in the simulation results.

From this chart it appears that the probability of winning the jackpot has little or no effect on the average winnings per show - whilst there is some spread of average winnings as we vary the jackpot probability for a given pointless answer probability, there appears to be no systematic relationship with the jackpot probability. The following chart, which presents the same data in a slightly different way, makes this even clearer.

View fullsize

That bobbling about as we move from left to right, increasing the jackpot probability for a given pointless answer probability, looks to be no more than random noise.

If we think about the game mechanics we can see why this must be true. Across any given series of shows in which the jackpot is won at least once, the total amount won will be equal to £1,000 x number of shows + £250 x number of pointless answers given in those shows. Whether this whole amount is won only in the very last show, or several pairs each win some proportion of it across a number of shows, the average winnings per show will simply be the sum of the available winnings divided by the number of shows. That average will certainly vary with the probability of providing a pointless answer, but not with the probability of the jackpot being claimed.

That latter, jackpot probability will, however, determine the variability of jackpot sizes, since more difficult to win jackpots will tend to be larger when they are eventually won. 

We can see this if we investigate the average jackpot size as we vary the probability of winning it.

View fullsize

Here we see quite a sharp decline in the average jackpot size as we assume that it is easier to win. At a probability of 10%, typical jackpots are in the £12,000 to £15,000 range, depending on how often pointless answers as given, whereas at 30% typical jackpots are only around £3,000 to £5,000.

There is also, of course, a systematic relationship between the average jackpot size and the probability of providing a pointless answer, as reflected in the ordering of the points for each given jackpot probability. As we'd expect, we generally see higher average jackpots when pointless answers are more probable.

Time then to investigate the question that led us here: on average, how many returning pairs will there be in a show?

Here, a moment's reflection on the game mechanics makes it obvious that the answer doesn't depend on the pointless answer and jackpot probabilities since:

• pointless answers do nothing but bolster the jackpot size; they have no effect on the pairs' probability of returning
• pairs are precluded from returning whenever they make the jackpot round, regardless of whether or not they win it

So, we can estimate the average number of returning pairs by averaging our simulation results across all 40 scenarios. That gives an answer of about 1.72 returning pairs per game.

We can also estimate probabilities for each of the four possible outcomes for the number of returning pairs:

• Prob(0 returning pairs) = 3%
• Prob(1 returning pair) = 33%
• Prob(2 returning pairs) = 52%
• Prob(3 returning pairs) = 11%

The 1.7 average returning pair figure and the probabilities of the different outcomes accord well with observation.

WHAT DOES IT MEAN FOR THE PRODUCERS?

In summary, we can glean from this analysis that:

• With, on average, 1.7 pairs returning each show, you'd need to plan for 2.3 new pairs per show, or about 126 pairs for a 55 show series.
• A typical pair of average ability can be expected to appear in just one show 25% of the time (when they make the jackpot round at their first appearance), and in two shows the other 75% of the time.
• The average winnings per show will be roughly equal to £1,000 + £5,000 x probability of a pointless answer. (Note that this is a conservative estimate that assumes every answer can be pointless, even if all that have gone before it have already been pointless. The true expectation will be lower.)
• The average jackpot will increase dramatically as the likelihood of winning it is reduced. Reducing the probability from 30% to 10% will roughly triple or quadruple the average jackpot - but also increase the average time between jackpots from about 3.3 shows to 10 shows, this average being given by 1/(jackpot probability)

WHAT NEXT?

It would be fascinating to compare the results found here by simulation with the actual results from Pointless history. I'll be hunting around the internet to see if anyone has created a database of individual show results, or has even generated summary tables.

If you're aware of any, please let me know.