Telling tall tales: how a truthful graph still misleads

0:00A family measures everybody, works out the average height, and builds a doorway exactly that tall. Before I go on, answer out loud: how many of them have to stoop? Five people — my own example, not a measurement of anybody. They add to eight hundred, and eight hundred shared between five is exactly a hundred and sixty. So the doorway is a hundred and sixty, and the tallest of them is thirteen centimetres taller.

0:32Two walk through freely, one fits exactly, and two have to duck. The average did not lie. It answered a question about the family, and they asked it about a person. That is the whole of this video, and it happens four more times. Now a real question, with real data. Are boys taller than girls? Two schools measured every child in three grades, and drew twelve dot plots on one shared scale.

1:03Those dots are the only record of the individual children, and not one carries a number, so I will not count them. What I do have is twelve averages, one beside each plot, and twelve averages is what I get to use. Every one of them is correct. So commit now, before the arrows go up. Boys, or girls? Take one row at a time, and draw an arrow towards whoever is taller.

1:33Grade six: the girls, by nearly three centimetres. Grade seven: the girls again, and now look at the margin. A hundred and forty one point eight three, against a hundred and forty one point eight. Three hundredths of a centimetre, standing for a whole class of children. Grade eight: the boys, by one and a half. The second school: girls, boys, girls. Four arrows point one way, two point the other, and one of the four turns on a third of a millimetre.

2:08Twelve correct numbers, and no answer. There is something far larger in those same twelve numbers. Line each grade up against the same grade in the other school. Grade six boys: a gap of fifteen centimetres. And it happens in all six rows. The second school is taller every single time. Just under seven centimetres at the closest, fifteen at the widest. The boy and girl gaps never once reached three.

2:40So the smallest school gap is more than twice the largest of those. The second school's Grade six children are taller than the first school's Grade eight children. And nothing here says why. The difference is in the data; the reason is not. So what am I allowed to write down? Not that girls are taller than boys. Not even that girls of this age are. This: in these two schools, in four of these six grades, the girls' average was the higher one.

3:15That sentence is dull, and it is true, and the exciting one was neither. The mistake has a shape, and the shape is reach. The data covers six classrooms; the claim covered every child alive. Two schools cannot settle it, and nor could two hundred, if they all sat in the same sort of place. You need data that reaches as far as your sentence does. So here is one that reaches further. A whole country, measured four times, ten years apart.

3:49Down the side, every age from five to nineteen. Across the top, the four survey years. Inside each year, two columns: boys and girls. Fifteen rows and eight columns is a hundred and twenty numbers, far too many to read out. So learn to point at it instead. One cell: the five-year-old boys in the first survey, a hundred and one point three. Now, six sentences people write about a table like this. Every one is worth testing.

4:25Sentence one. Both averages, at every age, went up between the first survey and the last. Fifteen ages and two columns is thirty comparisons, and there is no shortcut. Every one of the thirty is a rise, so the sentence is justified. It took thirty lookups to earn that. Now change three words. Went up every ten years. False, and the same thirty cells catch it. Six of them fell between the first two surveys, and the sixteen-year-old girls did not move at all.

5:01Same table, opposite verdict, because the sentence asked for something else. Sentence two. Thirteen-year-old girls in the first survey were taller than fourteen-year-old girls twenty years on. Two cells settle it, and it is refused by nearly five centimetres. Sentence three. Fifteen-year-old boys in the last survey were taller than sixteen-year-old boys in the first. A hundred and fifty nine point zero, against a hundred and fifty eight point nine.

5:37True, by one tenth of a centimetre. One millimetre. No bar on any graph could show you that, which is why you read the digits, not the picture. It really is that close: one age either way and the answer reverses. Sentence four. Right through from five to nineteen, the boys' average is above the girls'. This kind you refuse with a single cell, and the table hands you eleven.

6:07At age eleven in the last survey, the girls are ahead by one point six. They lead at five, ten, eleven and twelve, and somewhere in all four surveys. One counter-example sinks a claim about every. Eleven is generous. And the table says why, if you read the one-year steps. The boys' fastest year runs from twelve to thirteen: six point two centimetres. The girls' fastest runs from ten to eleven: five point eight.

6:41Two years earlier, which is exactly the window where they are ahead. Sentence five, and this is the one the whole topic is built to catch. Every thirteen-year-old girl is taller than every eleven-year-old girl. The averages certainly are, nine point one centimetres apart. But there is no thirteen-year-old girl anywhere in that table. There is an average, and an average is a summary of many. So here are five and five of my own, built so their averages are exactly the table's two numbers.

7:19The tallest eleven is a hundred and forty three; the shortest thirteen is a hundred and forty two point five. One crossing, and the sentence is gone, with both averages exactly right. This is the doorway again, wearing a table. Sentence six. Boys go on growing after nineteen. There is no row for twenty, so the table cannot say. What it does show is the growing running out: half a centimetre in the boys' last step, and nothing at all in the girls'.

7:56A hint is not an answer. Forwards by one step is safe enough: across the three decades they gained seven tenths, nine tenths, then one point four. So about a centimetre more next time is fair. Ten more decades of it is not. Backwards it fails outright. Near age five the steps are about six centimetres a year. Run six a year back from five and you get seventy seven at birth.

8:26A newborn is about fifty. One more habit, about the line up the side of a graph. Those nineteen-year-old boys gained three centimetres across thirty years. Drawn from zero, three out of a hundred and seventy is under two per cent of the picture. So start the line at a hundred and forty five instead. Now the same three centimetres is seven and a half per cent, and the change is visible.

8:56That is not a trick. It is the closer look the finding needed. It has a price: the window is forty centimetres where the whole scale was a hundred and seventy. So every difference inside it is drawn four and a quarter times bigger. Never eyeball a ratio off a picture that begins above zero. Last one. Twenty cities, ranked by how many buildings they have over a hundred and fifty metres.

9:27Three bars carry no number, on purpose. Estimate them. This one lies between the labelled bars either side, nearer the lower: about three hundred and fifteen. Say about. Two bars here are the same length and stand for eighty six and eighty two. Now two claims about the thirteenth row. Only twelve cities have more. Only seven have fewer. On the chart, both are counting. But only seven have fewer reaches off the chart, and it is wrong.

10:01This is a list of the twenty biggest, so every city left off has fewer still. The first survives for that reason: nothing left off a ranking beats its last row. Is the world's tallest building in the top city? The chart counts. It never measures. Three hundred buildings of a hundred and fifty one metres beat five, one of which is eight hundred. Read the quantity, not the ranking.