\documentclass[twoside,12pt]{article}

\setlength{\textwidth}{6.5 in}
\setlength{\textheight}{8.5 in}
\setlength{\topmargin}{0 in}
\setlength{\headheight}{0.25 in}
\setlength{\headsep}{0.25 in}
\setlength{\evensidemargin}{0 in}
\setlength{\oddsidemargin}{0 in}
\setlength{\parskip}{1ex}

\newcommand{\heading}[1]{\vspace{3ex} {\Large {\em #1}}}
\newcommand{\subheading}[1]{\vspace{3ex} {\large {\em #1}}}

\usepackage{graphics}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{latexsym}

\newtheorem{definition}{Definition}

\begin{document}

\begin{center}
{\Large Lecture 1 - Introduction to Probability} \\
6.042 - February 4, 2003
\bigskip
\end{center}

\footnotetext{Copyright \copyright 2003, Charles Leiserson, Srinivas Devadas, Eric Lehman.  All rights reserved.}

Let's play {\em Carnival Dice}!  You roll three dice, one colored red,
one green, and one blue.  If the number six comes up on at least one
of the dice, I pay you a dollar.  Otherwise, you pay me a dollar.
This is a perfectly fair game.  After all, if you roll only one die,
then the probability that a six comes up is 1 in 6.  So if you roll
{\em three} dice, then the probability that a six comes up is
obviously:

\begin{eqnarray*}
3 \cdot \frac{1}{6} & = & \frac{1}{2}
\end{eqnarray*}

\noindent Therefore, you have an even chance of winning!  Or do you?

\centerline{\rule{1.5in}{1pt}}

This game teaches two lessons.  First, {\em do not rely on intuition
to solve probability problems}.  For some reason, we all have an
indomitable belief in our ability to figure ``the odds''.  And perhaps
you were not misled by the faulty reasoning above.  But the
counterintuitive results and apparent paradoxes in probability
routinely trip up even experts, and they will bewilder you soon
enough.  The only safe way to proceed is to lean lightly on intuition
and heavily on the mathematical underpinnings of probability theory.
In this lecture, we'll set up a framework for solving probability
problems that will never lead you astray.

The second lesson is: don't play complicated betting games with a
mathematician.

\section{Analysis of Carnival Dice}

Typically, probability theory is used to model some sort of randomized
game, process, or experiment.  If our model is good enough, we can
mathematically derive additional properties of the game, process, or
experiment that are not immediately apparent.  For example, we'll
shortly derive the fact that playing a lot of Carnival Dice is going
to lose you a whole pile of money.  We'll analyze the game using a
four-step method that is a great way to approach a wide range of
probability problems.

\subsection*{Step 1:  Find the Sample Space}

A probabilistic model has several components.  The basic building
block is called an {\em outcome}.

\begin{definition}
An {\em outcome} of an experiment consists of the total information
about the experiment after it has been performed, including the values
of all random choices.
\end{definition}

\noindent Outcomes sometimes go by other names, such as {\em sample
point}, {\em atomic event}, or {\em elementary event}.  In Carnival
Dice, there are three random choices: the number that comes up on the
red die, the number on the green die, and the number on the blue die.
Therefore, an outcome in Carnival Dice is an ordered triple of
numbers, each in the range 1 to 6.  For example, $(6, 3, 4)$ is the
outcome where the red die comes up 6, the green die comes up 3, and
the blue die comes up 4.

\begin{definition}
The {\em sample space} for an experiment is the set of all
possible outcomes.
\end{definition}

A {\em tree diagram} is a handy tool that can help you determine the
sample space for an experiment.  Here's how to build one.  Begin with
a single node.  Create an outgoing branch to a new node for each value
of the first random choice in the experiment.  (In Carnival Dice, this
is the number that comes up on the red die.)  From each of these new
nodes, create an outgoing branch to a another new node for each value
of the second random choice.  Continue in this way until there are no
more random choices left in the experiment.  Each leaf of the
resulting tree corresponds to an outcome of the experiment.  The tree
diagram for Carnival Dice is shown in Figure~\ref{fig:lec1-tree1}.

\begin{figure}[htbp]
\centerline{\resizebox{!}{3.5in}{\includegraphics{lec1-tree1.eps}}}
\caption{The tree diagram for Carnival Dice.} \label{fig:lec1-tree1}
\end{figure}

\noindent As is often the case, we can not draw the tree diagram for
Carnival Dice completely, because there are too many outcomes.  But we
can get draw enough to see what the tree looks like and, consequently,
to be able to reason about the problem effectively.  In this case, the
tree diagram suggests that the sample space $S$ consists of all
possible ordered triples of numbers, each in the range 1 to 6:

\begin{eqnarray*}
S & = & \{ \ (1,1,1),\ (1,1,2),\ (1,2,1),\ \ldots,\ (6,6,5),\ (6,6,6) \ \}
\end{eqnarray*}

\noindent Using some mathematical notation\footnote{The English
language is full of words with ambiguous meanings, perhaps because it
was evolved for hunting brontosaurus rather than for discussing
intricate mathematical ideas.  To remove these ambiguities,
mathematicians communicate using a variety of odd symbols such as
$\times$, $\vee$, $\forall$, and $\exists$.  We shall use such symbols
often in $6.042$.  A useful table that explains their meaning appears
inside the front cover of the Rosen textbook and inside the head of
your TA.}, we can describe this sample space succinctly:

\begin{eqnarray*}
S & = & \{ 1, 2, 3, 4, 5, 6 \} \times \{ 1, 2, 3, 4, 5, 6 \} \times
        \{ 1, 2, 3, 4, 5, 6 \}
\end{eqnarray*}

\subsection*{Step 2:  Define Events of Interest}

Usually we are interested not in the probability of a single outcome
(such as rolling all ones), but in the probability of some more
complicated event (such as you winning in Carnival Dice).

\begin{definition}
An {\em event} is a subset of the sample space $S$.
\end{definition}

\noindent This is a neat definition.  Everything that could possibly
happen during an experiment can be characterized as a subset of the
sample space and is, therefore, an event.  For example,

\[
\{\ (1,1,2),\ (1,2,1),\ (2,1,1)\ \}
\]

\noindent is the event that the sum of the numbers shown on the three
dice is equal to four.  You winning in Carnival Dice is also an event.
We could describe it by explicitly listing all the outcomes that it
contains, as we did above, but a more compact description is also
possible:

\begin{eqnarray*}
W & = & \{ (x, y, z) \mid x = 6 \vee y = 6 \vee z = 6 \}
\end{eqnarray*}

\noindent In words, the right side of this equation says, ``the set of
all triples $(x, y,z)$ such that $x$ is 6 or $y$ is 6 or $z$ is 6''.

\subsection*{Step 3: Specify Outcome Probabilities}

So far we've only tried to capture the set of things than can possibly
happen in an experiment.  The next job is to assign likelihoods to
those things.

\begin{definition}
A {\em probability space} consists of a sample space $S$ and a {\em
probability function} $\Pr : S \mapsto \mathbb{R}$ such that:

\begin{enumerate}
\item $\forall s \in S \quad 0 \leq \Pr\{s\} \leq 1$
\item $\sum_{s \in S} \Pr\{s\} = 1$
\end{enumerate}
\end{definition}

\noindent 
The expression $\Pr\{s\}$ is read ``the probability of $s$''.
Intuitively, $\Pr\{s\}$ is the fraction of the time that $s$ is the
outcome of the experiment, if the experiment is repeated many times.
The two conditions on $\Pr$ listed above say, ``every outcome gets a
probability between 0 and 1'' and ``the sum of all outcome
probabilities is 1''.

Strictly speaking, $\Pr$ is an ordinary function, just like the
ubiquitous $f$ and $g$ in calculus.  But there are a couple caveats.
In this class, we'll usually write $\Pr\{x\}$ instead of $\Pr(x)$,
because often the thing in parentheses itself involves parentheses and
that gets to be a mess.  (That said, many authors use $\Pr(x)$ or
$\Pr[x]$ or even $\Pr x$.)  Furthermore, we'll shortly begin using the
$\Pr$ notation in all sorts of contexts in which one would never use
an ordinary function.

How do we assign a probability to each outcome in Carnival Dice?  That
is, for each outcome $s$ in the sample space, what is the right value
of $\Pr\{s\}$?  Ultimately, there is no indisputably correct answer.
We define $\Pr\{s\}$ in the way that best reflects our understanding
of the problem.  For example, if our experiment were flipping a
quarter, we might assign probability $\frac{1}{2}$ to the outcome
``heads'' and probability $\frac{1}{2}$ to the outcome ``tails''.  We
hope that is a good assumption, but it is an {\em assumption}, not a
empirically derived fact.  After all, real quarters probably aren't
perfectly balanced or aerodynamically uniform, and they might even
land on edge sometimes.  For most problems in this course--- but not
all--- you will be told what assumptions to make about outcome
probabilities.

Using a tree diagram, we can determine all outcome probabilities from
relatively {\em few} assumptions about the experiment.  Taking
Carnival Dice as an example, let's assume that the red die shows each
of its six faces with probability $\frac{1}{6}$.  We represent this by
writing $\frac{1}{6}$ on each branch out of the root node.  Let's
further assume that the green die also shows each of its faces with
probability $\frac{1}{6}$, regardless of the outcome of the red die.
So we write $\frac{1}{6}$ on each branch at the second level.
Finally, let's assume that the blue die shows each face with
probability $\frac{1}{6}$, regardless of the outcomes of the red and
green dice.  Again, we write $\frac{1}{6}$ on each branch at the third
level.  The resulting diagram is shown if Figure~\ref{fig:lec1-tree2}.

\begin{figure}[htbp]
\centerline{\resizebox{!}{3.5in}{\includegraphics{lec1-tree2.eps}}}
\caption{The tree diagram for Carnival Dice, annotated with probabilities.} \label{fig:lec1-tree2}
\end{figure}

Now {\em the probability of an outcome is the product of the
probabilities on the path from the root to that leaf}.  For example,
the path for outcome $(1,4,2)$ is highlighted in
Figure~\ref{fig:lec1-tree2}.  Using this rule, we conclude that the
probability of outcome $(1,4,2)$--- and every other outcome in the
sample space--- is:

\begin{eqnarray*}
\frac{1}{6} \cdot \frac{1}{6} \cdot \frac{1}{6} & = & \frac{1}{216}
\end{eqnarray*}

\noindent When all outcomes in the sample space are equally likely, as
is the case for Carnival Dice, the sample space is said to be {\em
uniform}.  Of course, many tree diagrams and sample spaces will not be
so simple and symmetric!

\subsection*{Step 4: Compute Event Probabilities}

We have now assigned each outcome a probability.  What remains is to
determine the probability of an {\em event}, such as the event that
you win a round of Carnival Dice.

\begin{definition}
The probability of an event $A$ is written $\Pr\{A\}$ and is equal to

\[
\sum_{s \in A} \Pr\{s\}
\]
\end{definition}

In words, the probability of an event is the sum of the probabilities
of the outcomes contained in that event.  For example, the probability
that the sum of the dice is four is:

\begin{eqnarray*}
\Pr\{ (1,1,2) \} + \Pr\{ (1,2,1) \} + \Pr\{ (2,1,1) \}
    & = & \frac{1}{216} + \frac{1}{216} + \frac{1}{216} \\
    & = & \frac{1}{72}
\end{eqnarray*}

Since each outcome in Carnival Dice has equal probability, determining
the probability that you win boils down to counting the number of
outcomes in the event:

\begin{eqnarray*}
W & = & \{ (x, y, z) \mid x = 6 \vee y = 6 \vee z = 6 \}
\end{eqnarray*}

\noindent Counting these outcomes is a bit tricky.  We'll look at a
log of counting techniques later, but for now we'll use an ad hoc
method.  If the red die is a six, then you win however the remaining
two dice come up.  This gives $6 \cdot 6 = 26$ winning outcomes.  If
the red die is not a six, but the green die is, then the red die could
come up 5 ways and the blue die 6 ways, giving $5 \cdot 6 = 30$ more
winning outcomes.  Finally, if the blue die is a six and the red and
green dices are not, then the red and green dice can come up in $5
\cdot 5 = 25$ ways.  Therefore, the total number of winning outcomes
is $36 + 30 + 25 = 91$.  Since each outcome has probability
$\frac{1}{216}$, the probability of the event that you win at Carnival
Dice is:

\begin{eqnarray*}
\Pr\{W\} & = & 91 \cdot \frac{1}{216} \\
    & = & \frac{91}{216} \\
    & \approx & 42\%
\end{eqnarray*}

We can check this answer by computing the probability that you {\em
lose}, which should be one minus the probability that you win.  The
number of losing outcomes in simply $5 \cdot 5 \cdot 5 = 125$, since
each die can come up only 5 different ways without being a six.
Therefore, the probability that you lose is:

\begin{eqnarray*}
125 \cdot \frac{1}{216}
    & = & \frac{125}{216} \\
    & \approx & 58\%
\end{eqnarray*}

\noindent This is consistent with our previous answer.  The probability
that you win ($91/216$) plus the probability that you lose ($125/216$)
is equal to 1, as it should be.

We can conclude that your odds of winning Carnival Dice are nowhere
near even.  In fact, they are worse than in any common casino game!
\end{document}