Estimating from statistical data, and scaling the estimate up

0:00Not every probability comes from an experiment you run. Most of the ones that matter come from records of what has already happened. A shop deciding what to stock, a company forecasting sales, an insurer setting a price, a researcher reading a survey: all reasoning from data somebody already collected. What is new is the last step, and it is one multiplication. You take a proportion measured on the people you asked and apply it to a much larger group you did not ask.

0:36That step is trivial to perform and expensive to justify, and this video is about the difference. Here is the case to work with. Fifty students in one class are asked, anonymously, for their favourite fruit. Twenty say mango, fifteen say apples, ten say bananas, five say grapes. First thing to check, before anything else: those four counts add to fifty. Nobody was left out and nobody counted twice, and that matters more than it looks.

1:11Now the probability that a student picked from this class prefers mango. Twenty of the fifty, which is two fifths, or nought point four, or forty per cent. And the four proportions add to exactly one, because every student gave exactly one answer. Pause on the word probability, because it has been smuggled in. A count over a total is just a proportion of the class. It becomes a probability only when you say how the student will be chosen.

1:45Choose at random and every one of the fifty carries the same weight, one fiftieth. Add up the weights of the mango answers and you get two fifths back. That is why the proportion is the probability: the picking rule put equal weight everywhere. Now pick the student sitting nearest the fruit bowl instead. That rule lands on the same person every time, so the answer is not two fifths.

2:13It is nought or one, depending who that is, and the same counts support no claim at all. Anonymous protects the answers; random is what makes the number mean something. Now the school, which has fifteen hundred students. If two fifths of them prefer mango, that is two fifths of fifteen hundred. Cancel first: fifteen hundred divided by fifty is thirty, times twenty is six hundred. Six hundred students, from a survey of fifty.

2:48That is the whole of the arithmetic. It assumes the fifty resemble the fifteen hundred in the one respect being measured. Not that they are the same people, only that mango is as popular among those you did not ask. Everything hangs on that sentence, and nothing in the survey can check it. Do the other three the same way. Apples, fifteen fiftieths, gives four hundred and fifty. Bananas, ten fiftieths, gives three hundred.

3:21Grapes, five fiftieths, gives one hundred and fifty. Add them: six hundred, four hundred and fifty, three hundred, one hundred and fifty. Fifteen hundred exactly, which is the whole school. That is a real check, the only one available on a scale-up, but it works only if you divide by the number you actually asked. Lose one respondent — twenty, fifteen, ten and four, all over fifty — and the estimates come to one thousand four hundred and seventy.

3:56The school does not add up, and that tells you a count went missing. So what is the six hundred? It is not a measurement of the school; nobody measured the school. It is not a guarantee, and it is not how many mangoes will be wanted. It is one number, two fifths, applied to fifteen hundred people, and only as good as the two fifths. Change one student's answer, so twenty-one say mango instead of twenty.

4:27The estimate moves from six hundred to six hundred and thirty. One person in that classroom carries thirty students of the school, because fifteen hundred divided by fifty is thirty. That is not an error in the method; it is the method. The obvious fix is to ask more people, and the standard advice is to take a larger sample and make it more representative. That is one sentence holding two different repairs, and only one of them fixes this.

5:02Watch what happens on a school small enough to check every possibility. Twelve students in three classes of four. In the surveyed class three of the four like mango; in each of the others, only one. So the school is five out of twelve; the surveyed class is three out of four. Now take every possible sample from that class. Average over every one of them, at every size, and it is three quarters every time.

5:33Exactly three quarters, including when you ask the whole class. The gap to the school's five twelfths is a third, and asking more of the same class never closes it at all. Now draw the sample from the whole school instead. Run the same exhaustive count: every possible sample of every possible size. The average answer is five twelfths, the school's own figure, at every size from one to twelve.

6:05And now something happens that did not happen before. Measure how far a typical sample strays from it, and that falls at every step as the sample grows. By six of the twelve it has more than halved; by twelve it is nothing. A bigger sample makes the answer steadier. A fairer sample makes it an answer about the right group. A bigger sample of the wrong group is a more precise measurement of the wrong group, and precision was not the problem.

6:39Three words are worth keeping apart, because they get used interchangeably. The population is the whole group the estimate is about: the fifteen hundred. The sample is the part you collected data from: the fifty. Sampling is the practice of choosing which part, and it is where all the difficulty lives. The sample sits inside the population, and it is one thirtieth of it. So the estimate speaks for one thousand four hundred and fifty people nobody asked.

7:14That is not an objection to sampling; asking everybody is usually impossible. It is a reminder of how much weight the sampling step is carrying. One more thing, and it is pure arithmetic. A bag of sweets is mixed and thirty taken at random: ten red, eight green, seven yellow, five blue. The bag holds six hundred. How many are yellow? Seven thirtieths of six hundred. Cancel first: six hundred over thirty is twenty, times seven is one hundred and forty, exactly.

7:53Now do it the other way. Seven divided by thirty rounds to nought point two three, and that times six hundred is one hundred and thirty-eight. Two sweets have vanished into the rounding. It goes the other way too: green is exactly one hundred and sixty, but rounding gives one hundred and sixty-two. And the damage is not rare. Count every way thirty answers could split four ways, round each share to two places, and more than a quarter no longer fill the bag.

8:28Cancel first, divide last. Notice something about that sweet bag that the fruit survey did not have. The thirty sweets were drawn out of the very bag being estimated. The sample is part of the thing being described, and the total is known. Compare a different survey: forty students, a random sample of a whole school of eight hundred. Fourteen say science club, eleven arts, nine sports, six debate. Arts is eleven fortieths, nought point two seven five, and sports scales to one hundred and eighty.

9:08That scale-up stands on firmer ground, and the difference is one phrase: the forty came from the school, not from one class. Same arithmetic, better foundation. The multiplication never tells you which situation you are in. Finally, the case where none of this applies. A tyre company records how far a thousand tyres ran before replacement. Under four thousand kilometres, twenty of them. Four to nine thousand, two hundred and ten.

9:44Nine to fourteen thousand, three hundred and twenty-five. Over fourteen thousand, four hundred and forty-five. Those add to a thousand, so the records are the whole population and no scaling happens at all. The relative frequencies are the answers: nought point nought two, and nought point four four five. For four to fourteen thousand you join two columns, two hundred and ten plus three hundred and twenty-five, which is nought point five three five.

10:18And the three answers add to one. Read those boundaries as continuous ranges, not whole kilometres, or four thousand exactly belongs to no column. When your data already covers everybody you are not estimating but counting, and the hard part of all this does not arise.