Independence, and why it is a different idea from having no outcome in common

0:00One card, drawn from a well shuffled deck. Face down on the table. Two questions about it. Is it a spade? Thirteen of the fifty two cards are spades, so that is a quarter. Is it an ace? Four of the fifty two, so that is one thirteenth. And there is exactly one card that answers yes to both — the ace of spades — so the chance of both is one over fifty two.

0:28Three numbers. Nothing unusual yet. But now do something with them. Suppose somebody turns the card halfway over, sees the rank, and tells you: it is an ace. You now know something. What has that done to the spade question? Work it out. Told that it is an ace, you are standing on a list of four cards. One of them is a spade. One in four. A quarter. Which is exactly what the spade question was worth before anybody said anything.

1:02The news arrived and the number did not move. Now run it the other way, because a viewer who only ever sees one direction will quietly decide this is a one way property. Told instead that it is a spade, you are standing on a list of thirteen cards. One of them is an ace. One in thirteen. Which is what the ace question was worth before anybody said anything either.

1:31So: knowing the suit tells you nothing about the rank, and knowing the rank tells you nothing about the suit. Neither piece of news moves the other one. That is the whole idea. Everything after this is how to say it precisely, and what goes wrong when people say it loosely. Here is the first way to write it down. Two events are independent when the chance of the first, given the second, equals the plain chance of the first — and the chance of the second, given the first, equals the plain chance of the second.

2:09Two equations, one for each direction. That is the definition in the shape it is usually met. And it comes with fine print, which matters more than fine print usually does. Both of those conditionals are fractions. The first one divides by the chance of the second event. The second one divides by the chance of the first. So the definition can only be written at all if neither of those is nought.

2:40Two side conditions, carried along explicitly. Keep them in view, because in about two minutes they are going to disappear, and that disappearance is the most useful thing in this topic. Take the first equation and clear its denominator. The chance of the first given the second is the chance of both, over the chance of the second. Setting that equal to the plain chance of the first and multiplying across gives this.

3:13The chance that both happen equals the chance of the first times the chance of the second. One line. No conditionals left anywhere in it. And now look at what went out of the room with them. Nothing is divided any more. So nothing has to be nonzero any more. The product form has no side condition at all. That is not a cosmetic difference. It means the second way of writing it applies to pairs the first way cannot even be written down for.

3:47Counted, on a space of three tossed coins with all two hundred and fifty six events and every ordered pair of them: there are sixty five thousand five hundred and thirty six pairs. Wherever both forms can speak they never once disagree. But five hundred and eleven of those pairs involve an event that cannot happen, and for every one of them the first definition is silent while the product answers.

4:16So the product is the usable definition, and the conditional form is the motivation. That is the order to hold them in. Dependence is then defined off the product, and it is defined in the laziest possible way, which is the right way. Two events are dependent when they are not independent. That is the whole of it. The chance of both fails to equal the product of the two chances, and there is nothing else to check.

4:49No degree, no direction, no mechanism. A verdict with two values. Which means every test in this topic has the same three steps. Work out the chance of the first. Work out the chance of the second. Work out the chance of both. Multiply two of them and look at the third. If a question ever seems to want more than that, it is asking something else. Now the confusion this whole topic exists to kill.

5:21Two events that cannot both happen. Roll a die: an even face and a face showing one. They share no outcome at all. They have, as people say, nothing to do with each other. Surely that is the most independent two events could possibly be. It is the opposite, and the reason is that those are two different kinds of statement about two different kinds of object. Having no outcome in common is a statement about sets. You look at the two lists, and you check whether anything appears on both. No arithmetic is involved. You never need to know a single probability to settle it.

6:03Independence is a statement about numbers. Three of them, and whether two of them multiply to the third. You could know the two sets perfectly and still not be able to answer. Draw them and the difference shows. Two circles that miss each other entirely — that is the first statement, and it is about the picture. Two circles that overlap, with three numbers written beside them — that is the second, and the picture cannot settle it.

6:35Independent events usually do overlap. The ace of spades was in both events at the start of this video. So they are different ideas. But there is a much sharper thing to say, and it is one line long. Take two events that share no outcome, and suppose both of them actually carry some probability. They cannot both happen, so the chance of both is nought. But neither chance is nought, and two numbers that are not nought never multiply to nought.

7:10So the chance of both is not the product of the chances. The equality fails. They are dependent. Always, with no exceptions, and the proof took four lines. This is usually stated as a warning to be memorised. It is a theorem, and it deserves to be read as one. It also has an intuition worth keeping. If two events cannot both happen, then learning that one of them occurred tells you the other certainly did not. That is the largest piece of news there is. Independence is the claim that there is no news at all. Those are as far apart as two conditions get.

7:53Over the three coin space: six thousand and fifty ordered pairs share no outcome with both chances nonzero, and not one of them is independent. Meanwhile seven thousand five hundred and nine independent pairs do share an outcome. And the nonzero condition is load bearing. Drop it, and five hundred and eleven pairs share nothing and are independent — every one of them involving an event that never happens. If the two conditions were the same condition, they would give the same answers. Here is a question that asks for one number and gets two.

8:38One event has chance a half. The chance that at least one of the two events happens is three fifths. Find the chance of the other event. Assume they cannot both happen. Then the chance of at least one is just the sum, so the second chance is three fifths minus a half. One tenth. Now assume instead that they are independent. The chance of at least one is the sum minus the chance of both, and the chance of both is now the product. Solve, and the second chance comes out at one fifth.

9:15One tenth against one fifth. Same question, same given numbers, two different answers, because the two assumptions are two different assumptions. Whenever a question hands you a union and asks for a missing piece, the first thing to find out is which of those two it is assuming. Three tests, run all the way through. A die. Let the first event be a multiple of three, so three or six, which is one third. Let the second be an even face, which is a half. They share the single face six, one sixth. A third times a half is a sixth. Independent.

9:59A die thrown twice. Odd on the first throw is a half, odd on the second is a half, odd on both is a quarter. A half times a half is a quarter. Independent — and here you knew it before the arithmetic, because the two throws are physically unconnected. That is worth noticing. The definition is a check, not an intuition, and this is the one case where the intuition was already right.

10:29Third, three coins, and three events on that one space. All three coins alike, a quarter. At least two heads, a half. At most two heads, seven eighths. First and second: they meet at three heads, an eighth, and a quarter times a half is an eighth. Independent. First and third: they meet at three tails, an eighth, against a product of seven thirty seconds. Dependent. Second and third: three eighths, against a product of seven sixteenths. Dependent.

11:03One sample space, three events, three tests, three different verdicts. Independence is not a property of the experiment. It is a property of the pair. Next, a result that sounds obvious and is worth proving anyway. If two events are independent, so are the first event and the opposite of the second. Draw the first event as a whole region, and cut it with the second. Every outcome of the first is either inside the second or outside it, and no outcome is both. So the first event splits in two, and its chance is the chance of the part inside plus the chance of the part outside.

11:50Rearrange. The part outside is the chance of the first, minus the chance of both. Now use independence on that second term: the chance of both is the product. So the part outside is the chance of the first, minus the chance of the first times the chance of the second. Take the common factor out, and what is left in the bracket is one minus the chance of the second — which is the chance of the opposite of the second.

12:22So the chance of the first and not the second is the chance of the first, times the chance of not the second. That is the product form. Independent. Run the same argument again with the roles exchanged and you get the first event's opposite against the second. Run it once more on that result and you get both opposites. Swept over every independent pair of the three coin space, all three versions hold at every single one — twenty four thousand and sixty checks, nought failures. Run the same sweep over the dependent pairs instead and every one of the three fails, every time.

13:08One immediate use, and it is the shape of question that comes up constantly. Two independent events. What is the chance that at least one of them happens? The long route uses the addition rule: the chance of the first plus the chance of the second, minus the chance of both. Independence lets you write that last term as a product, and then you factorise until it tidies. The short route skips all of it. At least one of them happens unless both of them fail. Both failing is the opposite of the first together with the opposite of the second — and we have just proved those two are independent, so that chance is a product.

13:54So the chance of at least one is one, minus the chance the first fails, times the chance the second fails. Two routes, the same expression. Try it: two people attempt a problem independently, with chances a half and one third. The chance it gets solved is one minus a half times two thirds, which is two thirds. And a die thrown three times, wanting an odd face at least once: one minus a half cubed. Seven eighths. Counting the successes directly would have taken three cases.

14:34Three events now, and a condition that most people never see fail. For three events to be independent you need four things: the three pairs each satisfying the product rule, and then the chance of all three together equal to the product of all three chances. Four conditions. And the obvious question is why the fourth is listed separately — surely if all three pairs work, the triple works. It does not. Here is a case, and it is built out of two coins.

15:09First event: a head on the first coin. Second: a head on the second coin. Third: the two coins agree. Each of those has chance a half. Take them in pairs. First and second meet only at two heads, a quarter, which is a half times a half. First and third meet only at two heads as well — a quarter again. Second and third, the same. All three pairs pass.

15:38Now all three at once. All three happen only on two heads. A quarter. And the product of the three chances is an eighth. A quarter is not an eighth. The fourth condition fails while the first three hold, so the three events are pairwise independent and not independent together. That is not a freak. On two coins there are two thousand seven hundred and forty four ordered triples of events that are neither impossible nor certain. Forty eight of them are pairwise independent. Not one of the forty eight passes the fourth condition.

16:20One more thing, because it is the sort of thing worth being able to spot. Suppose you are handed three numbers about a pair of events. The first has chance a half. The second has chance seven twelfths. The chance that at least one of the two fails is a quarter. That last one is the opposite of both happening. So the chance of both happening is one minus a quarter, which is three quarters.

16:49And now stop. Both events happening is a smaller thing than the first event happening. Its chance cannot be larger. But three quarters is larger than a half. The same contradiction arrives from the other end. The chance of at least one comes out at a half plus seven twelfths minus three quarters, which is one third — and one third is below the seven twelfths of an event contained in it.

17:18There is no pair of events anywhere with those three numbers. The question can be answered mechanically, by computing a product and comparing, without ever noticing that. Checking that the numbers you were given can coexist is a habit worth having. It costs one line. What to keep from all of this. Independence is an equation between three numbers: the chance of both events equals the product of the two chances. Test it that way. Three probabilities, one multiplication, one comparison.

17:58The conditional version — each event leaving the other's chance alone — is where the idea comes from and is how to think about it. But it needs both chances to be nonzero, and the product version needs nothing. Having no outcome in common is a different kind of statement altogether. It is about the two sets, not about three numbers. And with both chances nonzero the two conditions cannot hold together, because an empty overlap carries no probability and a product of two live numbers does.

18:35Independence survives complementing either event or both, which is what makes at least one of two independent events easy: one, minus the product of the two failures. And with three events, the three pairs are not enough. The fourth condition is a separate demand and it can fail on its own. Two events that cannot both happen are not unrelated. They are as related as two events ever get.