Massachusetts Institute of Technology Lecture 21 6.042J/18.062J: Mathematics for Computer Science November 19, 1996 Professors Nancy Lynch and Tom Leighton

Notes 21

1 Independence

Definition 1.1 An event A is independent of an event B if

P r(A|B) = P r(A).

In other words, knowing that B occurs doesn’t change the probability that A occurs.

For example, if we flip two coins, then the result of coin 1 is unrelated to that of coin two, thus the event “coin 1 is head” is independent of the event “coin two is head”.

Independent events are particularly nice to work with. This is because there is a very simple rule to compute the probability of their intersection.

Theorem 1.2 If A is independent of B, then

P r(A ∩ B) = P r(A) · P r(B)

Proof.

P r(A ∩ B) = P r(B) · P r(A|B) = P r(B) · P r(A)

Note that the product rule is symmetric in A and B; it is not by chance.

Theorem 1.3 If A is independent of B, then B is independent of A. In other words, independence is a symmetric relation.

Proof.

Pr(B|A) = \frac{P r(A ∩ B)}{P r(A)} = \frac{P r(B) · P r(A|B)}{P r(A)} = P r(B).

Thus it is appropriate to say that A and B are “independent of each other”. This makes it possible to define mutual independence of more than two events.

Definition 1.4 Events A_{1}, A_{2}, ⋯, A_{n} are mutually independent, if for all i, 1 ≤ i ≤ n and for all J, J ⊆ [1, n] − {i} we have

P r(A_{i}| ⋂_{j∈J} A_{j}) = P r(A_{i})

In other words, a collection of events is independent, if each of them is is independent of the intersection of any others. Equivalently, the above events are mutually indenpendent, if Π_{i=1}^{n}P r(A_{i}) = P r ∩_{i=1}^{n} (A_{i}).

For example, if we toss n coins and A_{i} is the event that the i-th coins is head, then these events are mutually independent, as P r(A_{i}) = 1/2 and this is true regardless of the results of other coins.

The following example is a little bit tricky. We flip three fair coins. Let

• A_{1}=coin 1 matches coin 2

• A_{2}=coin 2 matches coin 3

• A_{3}=coin 3 matches coin 1

Let’s see if these events are mutually independent. Clearly, P (A_{i}) = 2/4 = 1/2 for all i. Similarly, P (A_{i}∩A_{j}) = 2/8 = 1/4 for any i and j, if i ≠ j, as such an event occurs iff all three coins are head or all of them are tail. Thus P (A_{i})·P (A_{j}) = P (A_{i}∩A_{j}). However, these events are NOT mutually independent, as P (A_{1} ∩A_{2} ∩A_{3}) = 1/4 ≠ P (A_{1})P (A_{2})P (A_{3}) = 1/8. This illustrates how important it is to check all subsets when looking for mutual independence.

A weaker condition on independence is the following:

Definition 1.5 Events A_{1}, A_{2}, ⋯, A_{n} are pairwise independent, if for all i and j, i ≠ j, A_{i} and A_{j} are independent.

Thus mutual independence implies pairwise independence, but the converse is not true, as the previous example shows.

2 The Birthday Problem

Say there are m=80 people in the classroom. What is the probablility of the event A that there are two of them who have the same birthday?

There are N=365 days in a year. Therefore, we can describe the data in a vector of length m, say (b_{1}, b_{2}, ⋯, b_{m}) so that b_{i} denotes the birthday of the i-th person. Then we have N choices for each coordinate. The set of these vectors will be our sample space.

Definition 2.1 A sample space S is said to be uniform (or equiprobable) if Pr(S) is the same for all s ∈ S.

Theorem 2.2 If S is uniform and A ⊆ S, then P r(A) = |A|/|S|.

Proof. P r(A) = ∑_{a∈A} P r(a) = ∑_{a∈A} 1/|S| = |A|/|S|

In the sequel we assume that our sample space is uniform, that is, the birthdays of the people in the classroom are independent from each other. This means that for any s ∈ S, we have P r(s) = 1/N^{m} by the previous theorem.

Definition 2.3 The complement of the event A is S-A. It will be denoted by Ā.

Theorem 2.4

P r(Ā) = 1 − P r(A)

Proof. Obvious.

Now we are computing the number of sample points (vectors) with all different values to get the probability of the event Ā This is easy: we have N choices for the first person’s birthday, N-1 for the second person’s birthday, and so on, finally we have N-m+1 choices for the last person’s birthday. Thus

P r(Ā) = \frac{N(N − 1) ⋯ (N − m + 1)}{N^{m}}

This answers our question, but we are interested to know how large this number is. Sums are usually easier to deal with than products, thus we apply the well known inequality 1−x < e^{−x} with x = i/N. This yields:

P r(Ā) = Π_{i=1}^{m−1}(1 − \frac{i}{N}) ≤ Π_{i=1}^{m−1}e^{−i/N} =

= e^{− ∑_{i}^{m−1} − \frac{i}{N}} = e^{− \frac{(m−1)m}{2N}}

To get a lower bound, use the fact that 1 − x > e^{−x−2x^{2}/3} for 0 ≤ x ≤ 1/3. This yields

P r(Ā) ≥ Π_{i=1}^{m−1}e^{−i/N−2i^{2}/3N^{2}} = e^{− ∑_{i=1}^{m−1} i/N+2i^{2}/3N^{2}} ≥
≥ e^{− \frac{m(m−1)}{2N} + \frac{2m^{3}}{9N^{2}}}

If we use our upper bound in the case of N=365, m=80, then it follows that there is only 0.02 percent chance not to have two people in the classroom who have the same birthday. This is very surprising, as 80 is not even one fourth of 365.