Fair, unbiased, and memoryless: the gambler's fallacy

0:00A fair coin has just come up heads six times running. Almost everybody feels the same thing at this point: a tail is owed. Not merely possible next time, but somehow due, as though the coin had run up a debt. The feeling is worth taking seriously rather than laughing at, because the belief underneath it is half correct. Over a great many tosses the proportion of heads really does settle near a half.

0:30The mistake is in the mechanism: people imagine the coin repaying what it owes, and it does nothing of the kind. Before answering, ask what the coin would have to be able to do for the feeling to be right. It would need to keep a record: to know that six heads have gone by. And it would need to act on that record, leaning towards tails on the next throw to settle the account.

0:58Now put that beside what was assumed at the start, that the coin is fair, meaning neither side is favoured on any throw. One of those says the throw depends on nothing; the other says it depends on the last six. The intuition does not add something to the fair coin, it contradicts it. So the answer is flat: the probability of tails on the next toss is still one half.

1:26The coin has no memory, no record, and no way to act on one, and each toss starts from nothing. But stated on its own, that answer is unsatisfying, and it should be. If nothing is ever repaid, what makes the long run come out even? That question has an answer, it is arithmetic rather than assertion, and it is the whole of this video. Start from where the run left off: six heads in six tosses, so heads in every toss so far.

2:01The running proportion of heads is one point zero. Now keep tossing, and let the coin do the ordinary thing, which is split the rest about evenly. By a hundred tosses: six heads, plus half of the remaining ninety-four, which is fifty-three out of a hundred, or nought point five three. By a thousand tosses, six plus half of nine hundred and ninety-four, which is five hundred and three out of a thousand: nought point five nought three.

2:34By ten thousand, five thousand and three out of ten thousand: nought point five nought nought three. The proportion is walking towards a half, and it is walking there fast. Now look at what happened to the six heads themselves. In every one of those lines the heads run exactly three above half the tosses. Three above half at a hundred, three above half at a thousand, three above half at ten thousand.

3:05Nothing was cancelled and nothing was given back. Read it as heads minus tails and the answer is even blunter: the expected lead is six after the run, and it is still six after ten thousand tosses. What changed is the number underneath it. Three out of a hundred is three per cent; three out of ten thousand is three parts in ten thousand. The surplus is not repaid, it is diluted, and dilution is what makes long-run stability happen.

3:42One honest correction, because the surplus does sometimes disappear. A coin tossed long enough will wander back to level heads and tails at some point. Within twenty more tosses that happens about nineteen times in a hundred; within a hundred more, about fifty-five; within three hundred more, about seventy-three. But look at what is doing the work: that is chance wandering, and it wanders past level and out the other side just as readily.

4:16It is not the coin settling an account, and you cannot use it to predict a single throw. The expected lead stays at six the whole time. Same reasoning, different object. You are playing a board game with an ordinary six-sided die and have just rolled three sixes in a row. A fourth six now feels out of the question. Count it out instead of feeling it out. Four rolls of a die can fall in one thousand two hundred and ninety-six ways, all equally likely.

4:51Six of those ways open with three sixes, and exactly one of the six carries a fourth six. So the fourth roll is a six one time in six, which is precisely what it was before you touched the die. The three sixes were unusual, one run in two hundred and sixteen, and being unusual is not the same as being spent. The word for this is independent, and it gets used loosely, so here is a working definition.

5:24Two trials are independent when knowing how the first came out does not change the probabilities for the second. Notice what that does not say: it does not say the two have nothing to do with each other. Two tosses of the same coin, in the same hand, in the same minute, are as connected as two events could be in every way except the one that counts, and they are still independent.

5:52Independence is about probabilities changing, not about subjects being unrelated. Two more words, and keeping them apart is the point. A coin is fair when it is symmetrical: the two faces are alike enough that neither is favoured, so there is no physical reason for it to land one way more often. That is a claim about the object, and you could in principle check it by cutting the coin in half and looking.

6:24Unbiased is the name for that same property. A random toss is something else: the coin drops of its own accord, with nothing and nobody steering it. That is a claim about the procedure. A perfectly symmetrical coin placed carefully on the table heads-up satisfies the first and fails the second, and you get heads every time. Here is a trap worth walking into deliberately. Imagine building the coin the fallacy wants: after every toss it leans against whichever side is ahead.

7:02Run it, and something strange comes out. Toss by toss it is perfectly unbiased, favouring heads exactly half the time on the first toss, the second, the third, and every toss after. And yet after six heads it gives heads only one time in eight. So being unbiased and having no memory are two different properties, and a coin can have the first without the second. It is worth adding that unbiased has to mean every toss: a coin that is even on its first throw and leans three to one afterwards is not an unbiased coin.

7:44Now the question that catches people, and it catches them because of where it sits. You toss a coin twenty times and write down every result. You work out how often heads came up, how often tails came up, the proportions, all from your own record. Then you are asked for the probability of tails on one more toss. After twenty tosses of writing down real frequencies, the tempting answer is whichever side is behind.

8:16The answer is one half. Twenty tosses can show twenty-one different tallies, from no heads at all to all twenty, and every single one of them leaves the next toss at a half. So does the order they came in. Which leaves one honest question: is a long run evidence of anything at all? Yes, but not about the next toss. Six tosses fall sixty-four ways and exactly one is all heads, so a fair coin gives six heads about one attempt in sixty-four: unusual, nowhere near impossible.

8:56Suppose you suspect the coin leans, favouring heads three times in four. Six heads support that coin over a fair one by seven hundred and twenty-nine to sixty-four, under twelve to one, and twelve to one is a suspicion rather than a verdict. A hundred heads would be another matter entirely. So the run tells you about the coin, and nothing about the next toss once you have decided what the coin is.

9:25One last thing, since this is a video about not being fooled by numbers. A sixth is nought point one six six six, going on forever, so to three places it rounds to nought point one six seven, and as a percentage, sixteen point seven. Written as nought point one six six, or sixteen point six, it has been cut short rather than rounded: a tenth of a point further from the truth.

9:55The rule underneath all of it fits in one line: nothing is owed, and the long run needs no debt to settle.