\documentstyle[12pt,twoside,amsfonts,eepic,psfig]{article}
\input{macros-course-lectures}
\parskip1.5ex
\parindent0pt


\begin{document}
\lecture{notes19}{Notes 19}{November 9, 1996}
\newcommand{\binom}[2]{{#1 \choose #2}}
\section{Probability}

For the rest of the semester, we'll be talking about our final
topic of the course --- probability.  This is a subject that
comes up, in one form or another, in most areas of computer science.
It's important in the design of algorithms, coding theory, cryptography,
fault tolerance \ldots\ it also comes up in everyday life, in things
like political polls and weather prediction.  

Unfortunately, probability is one of the least understood and most
abused subjects in all of mathematics --- which is funny, since it's
really very easy, provided you follow a few basic rules when analyzing
a problem.  People get into trouble when they rely too heavily on their
intuition at the expense of these rules.

\section{Let's Make a Deal}

We'll start with a problem, the ``Monty Hall Problem,'' that recently 
became sort of notorious.  Just for a little background, Monty Hall
was the host of a game show called {\em Let's Make a Deal}, which,
depending on whom you ask, was either a great TV show or one of the
all-time low points in the history of televsion.
A typical scenario on {\em Let's Make a Deal} was the following:
on stage are three doors, numbered $1$, $2$, and $3$; and Monty
asks you to pick one of the doors.  Behind one of 
these doors is a new car, and behind the other two is something terrible,
like a donkey, or a goat.

So you pick one door.  Now Monty, who knows what's behind all the doors,
selects a door you didn't pick that has a goat behind it, and opens it.
Then he asks you,
\begin{quote}
{\em Do you want to switch doors?}
\end{quote}
That is, do you want to stay with the original door you picked, or
switch to the other unopened door?
We'll spend the rest of this lecture trying to answer this question.

This was the subject of an ``Ask Marilyn''
column in {\em Parade} Magazine a few years ago.  Marylin wrote that
your chances of winning were $2/3$ if you switched --- because if you
switch, then you win if the car was originally behind either of
the two doors you didn't pick.  Now, Marylin is listed in the
{\em Guiness Book of World Records} as having the world's highest IQ,
but for this answer she got a tidal wave of angry mail, much of it
from people with Ph.D.'s in mathematics, telling her she was wrong,
and that the answer was 1/2 --- because after Monty opens a door, the
car is now equally likely to be behind one of the two remaining doors.

It turned out that Marilyn was right.
But both of the arguments above are based on intuition, and the answer is
actually pretty subtle.  So rather than try to come up with our own 
explanation in words, let's introduce some of the definitions of probability.

\section{The Sample Space}

The key to this problem, and almost all probability problems, is to
look carefully at the space of all possible outcomes.

\begin{defin}
The {\em sample space} for a problem (or experiment) is the set consisting
of all possible outcomes.
\end{defin}

\begin{defin}
An {\em outcome} (a.k.a {\em sample point} or {\em atomic event}) of 
an experiment consists of the total information about the experiment
after it has been performed.  (This includwes the value of all random
choices that were made.)
\end{defin}

Let's re-phrase our main question a little.
\begin{quote}
{\em What is the probability of winning if you switch doors?}
\end{quote}
Is it $2/3$ as Marylin said, or $1/2$ as all those outraged readers
said, or neither?

So let's work out the sample space for the  problem.  An outcome of
the experiment consists of 
\begin{enumerate}
\item The number of the door containing the car.
\item The number of the door chosen by the contestant.
\item The number of the door opened by Monty.
\end{enumerate}
So a sample point is a $3$-tuple of numbers, containing these three
quantities.  For example, in the sample point $(2,1,3)$, the 
contestant won, because the car was behind door $2$, she picked $1$,
Monty opened $3$, and therefore she switched to $2$.

Not every $3$-tuple is a sample point.  For example, $(1,2,1)$ is
not a sample point because Monty never opens the door containing the prize.
Similarly, $(1,2,2)$ is not a sample point, because he never opens
the door you chose, either.

A good way to draw the sample space $S$ is in the form of a tree --- first
a branch for the first component of the $3$-tuple, then a branch for
the second, and then a branch for the third.  Here's a picture of $S$.

\begin{figure}[htb]
\begin{center}
\setlength{\unitlength}{0.0075in}
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(284,354)(0,-10)
\path(0,165)(80,265)(180,305)(260,325)
\path(180,305)(260,285)
\path(80,265)(180,265)(260,265)
\path(80,265)(180,225)(260,225)
\path(0,165)(80,165)(180,185)(260,205)
\path(80,165)(180,165)(260,185)
\path(180,165)(260,145)
\path(80,165)(180,125)(260,125)
\path(0,165)(80,65)(180,95)(260,95)
\path(80,65)(180,65)(260,65)
\path(80,65)(180,25)(260,45)
\path(180,25)(260,5)
\put(15,200){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(40,170){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(35,125){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(135,290){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(150,265){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(140,245){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(130,180){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(155,160){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(125,130){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(130,85){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(150,65){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(130,45){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(230,320){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(235,295){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(235,265){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(235,225){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(225,200){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(235,180){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(235,150){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(225,125){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(235,95){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(235,65){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(225,35){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(235,10){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(270,320){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,280){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,255){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,220){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,200){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,180){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,140){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,120){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,90){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,60){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,35){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,0){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\end{picture}
\end{center}
\caption{The sample space for our problem}
\label{space}
\end{figure}

Now, what we really care about is whether each outcome leads to a
win (W) or a loss (L) --- so we label each sample point accordingly.

Let $S_W \subseteq S$ denote the subset of winning outcomes.
We say that $S_W$ is the {\em event} that the contestant won.

\begin{defin}
An {\em event} is a subset of the sample space.
\end{defin}

Any subset of the sample space is an event.

\section{The Probability of Winning}

What we really want is the probability of winning.  In our space $S$,
we have six winning outcomes and six losing outcomes.  Does this mean
that the probability of winning is $1/2$?  No!  At least, not
necessarily!  Some outcomes may be more or less likely to occur 
than others, so we have to be careful.  (Embedded in here is the mistake
most of Marylin's readers were probably making --- though of course
we can't know for sure.)

In general, the only way to determine the probability of an event
is to define the sample space associated with the problem.

\begin{defin}
A {\em probability space} consists of a sample space $S$ and a
probability function $Pr : S \rightarrow \reals$, such that
\begin{enumerate}
\item For all $s \in S$, $0 \leq Pr(S) \leq 1$.
\item $\sum_{s \in S} Pr(s) = 1.$
\end{enumerate}
\end{defin}

So a probability space is a sample space with a probability for each
outcome.  Assigning the correct probabilities to outcomes is 
the most important part of analyzing one of these problems.

\begin{defin}
For any event $S' \subseteq S$, $$Pr(S') = \sum_{s \in S'} Pr(s).$$
\end{defin}

That is, to determine the probability of an event, we just have
to add up the probabilities of the outcomes it contains.

To make up probabilities for our outcomes, we'll make some reasonable
but arbitrary assumptions.  Namely, 
\begin{enumerate}
\item The prize is placed behind any door with equal probability ($= 1/3$).
\item The contestant picks any door with equal probability, regardless
of where the prize was placed ($= 1/3$).
\item Now, if the contestant picks the door with the prize, the Monty
has a choice of which of two doors to open, and he picks among them
with equal probability ($= 1/2)$.  Otherwise, he picks the only door
he is allowed to open under the rules.
\end{enumerate}

Given this, we can label each outcomes with a probability, as in the
next figure.  Basically, a $3$-tuple that results in Monty having a
choice of doors to open has probability 
$$\frac{1}{3} \cdot \frac{1}{3} \cdot \frac{1}{2} = \frac{1}{18},$$
while a $3$-tuple that results in Monty having no choice of which
door to open has probability
$$\frac{1}{3} \cdot \frac{1}{3} \cdot 1 = \frac{1}{9}.$$

\begin{figure}
\begin{center}

\setlength{\unitlength}{0.0075in}
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(326,354)(0,-10)
\path(0,165)(80,265)(180,305)(260,325)
\path(180,305)(260,285)
\path(80,265)(180,265)(260,265)
\path(80,265)(180,225)(260,225)
\path(0,165)(80,165)(180,185)(260,205)
\path(80,165)(180,165)(260,185)
\path(180,165)(260,145)
\path(80,165)(180,125)(260,125)
\path(0,165)(80,65)(180,95)(260,95)
\path(80,65)(180,65)(260,65)
\path(80,65)(180,25)(260,45)
\path(180,25)(260,5)
\put(15,200){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(40,170){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(35,125){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(135,290){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(150,265){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(140,245){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(130,180){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(155,160){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(125,130){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(130,85){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(150,65){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(130,45){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(230,320){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(235,295){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(235,265){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(235,225){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(225,200){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(235,180){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(235,150){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}3}}}}}
\put(225,125){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(235,95){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(235,65){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(225,35){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}1}}}}}
\put(235,10){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}2}}}}}
\put(270,320){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,280){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,255){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,220){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,200){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,180){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,140){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,120){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,90){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,60){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}W}}}}}
\put(270,35){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(270,0){\makebox(0,0)[lb]{\smash{{{\SetFigFont{10}{12.0}{rm}L}}}}}
\put(305,320){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/18}}}}}
\put(305,280){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/18}}}}}
\put(305,255){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/9}}}}}
\put(305,220){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/9}}}}}
\put(305,200){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/9}}}}}
\put(305,120){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/9}}}}}
\put(305,90){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/9}}}}}
\put(305,60){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/9}}}}}
\put(305,35){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/18}}}}}
\put(305,0){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/18}}}}}
\put(305,180){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/18}}}}}
\put(305,140){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}1/18}}}}}
\end{picture}
\end{center}
\caption{The probability space for our problem}
\label{prob.space}
\end{figure}

Now it's (finally) easy to go through the space, add up the probabilities
of the winning outcomes, and thereby determine the probability of
the event $S_W$, that the contestant wins by switching doors.

The answer is 
$$6 \cdot \frac{1}{9} = \frac{6}{9} = \frac{2}{3}.$$
So Marilyn was right, and all those math Ph.D.'s who reasoned with her,
pleaded with her, and even threatened her, were wrong.

Tune in next week for another edition of {\em Let's Make a Deal} \ldots\


\end{document}
