\documentclass[twoside,12pt]{article}

\setlength{\textwidth}{6.5 in}
\setlength{\textheight}{8.5 in}
\setlength{\topmargin}{0 in}
\setlength{\headheight}{0.25 in}
\setlength{\headsep}{0.25 in}
\setlength{\evensidemargin}{0 in}
\setlength{\oddsidemargin}{0 in}
\setlength{\parskip}{1ex}

\usepackage{graphics}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{latexsym}

\newenvironment{comment}[1]{}{}
\newenvironment{proof}{\noindent {\em Proof.}}{$\Box$}

\newtheorem{theorem}{Theorem}
\newtheorem{definition}{Definition}
\newtheorem{claim}{Claim}
\newtheorem{fact}{Fact}
\newtheorem{mrule}{Rule}

\newcommand{\beqn}{\begin{eqnarray*}}
\newcommand{\eeqn}{\end{eqnarray*}}
\newcommand{\pr}[1]{\Pr\left\{#1\right\}}
\newcommand{\ex}[1]{\text{Ex}\left[#1\right]}
\newcommand{\true}{\text{T}}
\newcommand{\false}{\text{F}}

\begin{document}

\begin{center}
{\Large Lecture 21 - Random Processes} \\
6.042 - May 6, 2003 \\
\bigskip
\end{center}

\section{The Duel}

Two gunfighters take turns shooting at each other.  Each shot hits
with probability $p = \frac{1}{6}$ and misses with probability $1 - p
= \frac{5}{6}$.  What is the probability that the first gunfighter to
shoot wins the duel?

We can approach this problem by the usual method, drawing a tree
diagram.  In contrast to previous problems, this tree diagram is
infinite.  As a practical matter, we can treat an infinite tree in the
same way that we treat large finite trees: draw just enough to make
the overall structure apparent.

\bigskip
\centerline{\resizebox{!}{2.5in}{\includegraphics{lec21-tree1.eps}}}
\bigskip

\noindent The probability that the first shooter wins is the sum of
the probabilities of the outcomes in that event:

\beqn
\pr{\text{first shooter wins}}
	& = &	p + (1-p)^2 p + (1-p)^4 p + (1-p)^6 p + \ldots \\
	& = &	p \sum_{k=0}^{\infty} (1-p)^{2k} \\
	& = &	p \cdot \frac{1}{1 - (1-p)^2} \\
	& = &	\frac{1}{2 - p}
\eeqn

\noindent In the second step, we pull out a $p$ from every term.  This
leaves a geometric sum.  The third step uses the formula for an
infinite geometric sum with ratio $r = (1-p)^2$:

\beqn
\sum_{k=0}^{\infty} r^k & = & \frac{1}{1-r} \hspace{0.5in} (\text{for $|r| < 1$})
\eeqn

\noindent The last step is simplification.  Since each shot hits with
probability $p = \frac{1}{6}$, the first gunfighter wins with
probability $\frac{6}{11}$.

\subsection{A Fixed-Point Approach}

There is a versatile, alternative approach to the gunfighter problem.
The key idea is to identify subtrees in the tree diagram that strictly
contain themselves.  With infinite trees, this is possible!  For
example, notice that the subtree shown below in bold is a complete
copy of the tree as a whole.

\bigskip
\centerline{\resizebox{!}{2.5in}{\includegraphics{lec21-tree1b.eps}}}
\bigskip

\noindent This reflects the fact that duel is in the same state after
two misses that it in was at the very beginning.  In each case, both
gunfighters are alive and the first gunfighter is about to shoot.

Let's introduce a variable $x$ to denote the probability that the
first gunfighter eventually wins, given that the duel is in this
state.  Then we can read an equation in $x$ directly from the tree:

\beqn
x & = & p + (1 - p)^2 x
\eeqn

\noindent The $p$ appears because the first gunfighter may hit
immediately.  Otherwise, with probability $(1-p)^2$, both gunfighters
miss and the duel is back in its original state.  If this happens,
then the first gunfighter eventually wins with probability $x$.  This
gives the $(1-p)^2 x$ term.  The only remaining possibility is that
the first gunfighter misses and the second hits.  However, the first
gunfighter is dead in that case, so this possbility adds nothing to
$x$, the probability that the first gunfighter wins.

Solving the equation above for $x$ gives:

\beqn
x & = & \frac{1}{2 - p}
\eeqn

\noindent This is the probability that the first gunfighter wins,
given that he is about to shoot.  Since this is the situation at the
start of the duel, this is also the overall probability that the first
gunfighter wins.  Happily, this is the same answer that we found
before!

\section{Infinite Problems}

Infinite processes and infinite sample spaces introduce theoretical
complications whose resolution lies beyond the scope of this course.
We'll note some of those complications and show you some practical
methods for working out problems involving infinity, but we're going
to gloss over the deeper theoretical issues.

For example, let's work out the sample space for the duel.  Let $M$
indicate a missed shot, and let $H$ indicate a hit.  Then the sample
space is:

\[
\{ H,\ MH,\ MMH,\ MMMH,\ \ldots \} \cup \{ MMM\ldots \}
\]

\noindent This says that a duel either consists of a sequence of
misses followed by a hit or else an infinite sequence of misses.  The
last outcome is the most puzzling.  Each of the finite sequences
corresponds to a leaf vertex in the tree diagram, as usual.  However,
the outcome $MMM\ldots$ corresponds to an {\em infinite path} through
the tree rather than to a leaf.  And there is another oddity.  In
principle, the gunfighters could go on missing forever, much as you
could flips heads forever on a fair coin, so $MMM\ldots$ must be
included as an outcome in the sample space.  But the probability of
that outcome is zero!

This leads to a weird conclusion: an event with probability zero {\em
can} occur.  For example, suppose that we flip a fair coin forever.
The probability of generating any particular sequence of heads and
tails (say, HTHTHTHTHTHT\ldots) is zero.  And yet we are bound to get
some sequence.  Therefore, perversely, {\em we are certain to generate
a sequence that has zero probability of being generated}.

Maybe you should revisit those childhood plans to be a firefighter.

\subsection{Kolmogorov's Axioms}

The Russian mathematician Kolmogorov proposed a retooling of the
foundations of probability to cope with such problems.  In this
course, we regard $\Pr$ as a function defined on {\em outcomes},
individual elements of the sample space.  In Kolmogorov's
reformulation, $\Pr$ is a function defined on {\em events}, entire
subsets of the sample space.  In particular, three properties must
hold for every valid probability function $\Pr$ over a sample space
$S$:

\begin{itemize}

\item $\Pr(S) = 1$

\item For all events $A \subseteq S$, $\pr{A} \geq 0$.

\item If $A_1, A_2, A_3, \ldots$ is a sequence of disjoint events, then:

\beqn
\pr{A_1 \cup A_2 \cup A_3 \cup \ldots}
	& = & \pr{A_1} + \pr{A_2} + \pr{A_3} + \ldots
\eeqn

\end{itemize}

One implication of Kolmogorov's axioms is that {\em there is no way to
pick a natural number uniformly at random}.  The proof is by
contradiction.  Suppose that there is a way to pick a natural number
uniformly at random.  This means that over some sample space $S$, we
can define a random variable $X$ that takes on each natural number
with some fixed probability $\epsilon$.  Then we can reason as
follows:

\beqn
1	& = &	\pr{S} \\
	& = &	\sum_{k=0}^\infty \pr{X = k} \\
	& = &	\sum_{k=0}^\infty \epsilon \\
	& = &	\left\{
\begin{array}{ll}
0 & \text{if $\epsilon = 0$} \\
\infty & \text{if $\epsilon > 0$}
\end{array}
\right.
\eeqn

\noindent The first step uses Kolmogorov's first property.  The second
step uses the third property, since the events $X = k$ partition the
sample space.  The third step uses the fact that the random variable
$X$ is equal to each natural number with probability $\epsilon$.  In
the final step, we evaluate the sum.  The result is either 0 (if
$\epsilon = 0$) or infinity (if $\epsilon > 0$).  By the second
property, these are the only two possibilities.  Therefore, we have a
contradiction.  Thus, there does not exist a way to pick a natural
number uniformly at random.

\subsection{``Prussian'' Roulette}

Suppose that a lunatic is playing a variant of Russian roulette in
which the gun becomes progressively less likely to fire.  In
particular, let's set up the probabilities so that the lunatic dies
after exactly $k$ shots with probability $1 / 3^k$.  Our probability
tree might look like this:

\bigskip
\centerline{\resizebox{!}{2.5in}{\includegraphics{lec21-tree2.eps}}}
\bigskip

\noindent The probabilities on the edges are constructed just to make
the probabilities on the leaves work out.  If the tree continues in
this way, then the probability that the lunatic {\em never} dies is:

\beqn
1 - \left(\frac{1}{3} + \frac{1}{9} + \frac{1}{27} + \ldots \right)
	& = &	1 - \frac{1}{3} \cdot \sum_{k=0}^{\infty} \frac{1}{3^k} \\
	& = &	1 - \frac{1}{3} \cdot \left( \frac{1}{1 - 1/3} \right) \\
	& = &	\frac{1}{2}
\eeqn

\noindent Therefore, there with probability $\frac{1}{2}$, this
process {\em never} terminates!

\section{The Duel Revisited}

Let's now return to the two gunfighters who take turns shooting at
each other, hitting on each shot with probability $p = \frac{1}{6}$.
What is the expected number of shots fired during the gunfight?

\subsection{An Expectation Formula}

To determine the expected duration of the duel, we'll use a nifty new
formula for the expectation of a natural-valued random variable.

\begin{theorem}
Let $X$ be a random variable that takes on values in $\mathbb{N}$.
Then:

\beqn
\ex{X} & = & \sum_{k=0}^{\infty} \pr{X > k}
\eeqn
\end{theorem}

\begin{proof}
We start with the left side of the equation and reason as follows.

\[
\begin{array}{rcrcrcrcl}
\ex{X}	& = &	1 \cdot \pr{X = 1} & + &
		2 \cdot \pr{X = 2} & + &
		3 \cdot \pr{X = 3} & + & \ldots \\
	& = &	\pr{X = 1} & + & \pr{X = 2} & + & \pr{X = 3} & + & \ldots \\
	&   &              & + & \pr{X = 2} & + & \pr{X = 3} & + & \ldots \\
	&   &              &   &            & + & \pr{X = 3} & + & \ldots \\
	&   &              &   &            &   &            & + & \ldots \\
	& = &	\multicolumn{7}{l}{\pr{X > 0} + \pr{X > 1} + \pr{X > 2} + \ldots}
\end{array}
\]

\noindent The first step uses the definition of expected value.  In
the second step, we expand each term $k \cdot \pr{X = k}$ to a {\em
column} of terms containing $k$ copies of the term $\pr{X = k}$.  In
the third step, we collapse each {\em row} of terms $\pr{X = k + 1} +
\pr{X = k + 2} + \ldots$ to a single term $\pr{X > k}$.  The last
expression is equal to the right side of the equation, so the theorem
is proved.
\end{proof}

\subsection{Applying the Formula}

Let the random variable $X$ equal the number of shots fired in the
duel.  Then we have:

\beqn
\ex{X}	& = &	\sum_{k=0}^{\infty} \pr{X > k} \\
	& = &	\sum_{k=0}^{\infty} (1 - p)^k \\
	& = &	\frac{1}{1 - (1-p)} \\
	& = &	\frac{1}{p}
\eeqn

\noindent The first step uses the new expectation formula.  For the
second step, note that more than $k$ shots are fired only if the first
$k$ shots are misses, which happens with probability $(1 - p)^k$.  We
use the formula for the sum of a geometric series in the third step,
and the final step is simplification.  Since our gunfighters hit with
probability $p = 1/6$, the expected number of shots fired is $1 / p =
6$.

Restating this conclusion in more general terms gives a fact that is
very useful and worth remembering.  Suppose that there is a sequence
of independent trials, each of which succeeds with probability $p$ and
fails with probability $1 - p$.  Then the expected number of trials
needed to obtain a success is $1 / p$.

\section{A Good Bet}

A big part of our job as 6.042 staff is to constantly deceive, cheat,
and abuse students.  But this time we'll make an exception!  How much
would you pay us up front for the privilege of playing the following
game?  You flip a coin until you get heads.  Then we pay you $2^k$
dollars, where $k$ is the total number of times you flipped.  For
example, if you flipped TTH, then we would pay you \$8.

Let the random variable $X$ be the amount of money that we pay you, if
you play this game.

\beqn
\ex{X}
	& = &	\sum_{k=1}^{\infty} 2^k \cdot \pr{\text{you flip $k$ times}} \\
	& = &	\sum_{k=1}^{\infty} 2^k \cdot \frac{1}{2^k} \\
	& = &	\sum_{k=1}^{\infty} 1 \\
	& = &	\infty
\eeqn

\noindent The first step uses the definition of expectation.  In the
second step, note that you flip exactly $k$ times only if you flip
$k-1$ tails followed by a head, which happens with probability $1 /
2^k$.  In the remaining two steps, we simplify and observe that the
sum diverges.

This calculation shows that your expected payoff is {\em infinite}!
At least in theory, this is the deal of a lifetime; you should be
willing to pay everything you've got for just one chance to play this
game!

\subsection{Some Practical Considerations}

The theoretical analysis of this problem overlooks some important
practical considerations.

This game looks good on paper because you have a tiny probability of
winning a vast amount of money.  Mathematically, this makes the
expected payoff infinite.  But, in practice, winnings obey a law of
diminishing returns.  For example, would you rather have a million
dollars upfront or a 1-in-a-1000 chance of winning a billion dollars?
(And, correspondingly, a 999-in-a-1000 chance of getting nothing.)
Most people would probably take the million, figuring that while a
billion dollars is a 1000 times {\em more}, it is unlikely to make
them 1000 times {\em happier}; either way, you're rich!  So why not
take the 100\% chance rather than the 0.1\% chance?

Futhermore, there isn't an infinite amount of money in the world and
certainly isn't an infinite amount in our pockets.  Suppose
that you suspect we actually couldn't pay you more than $2^{20}
\approx$ a million dollars, no matter how many tails you flip.  In
that case, we can compute the expected value of the game as follows:

\beqn
\ex{X}
	& = &	\left(\sum_{k=1}^{20} 2^k \cdot
		\pr{\text{you flip $k$ times}}\right) +
		\left(2^{20} \cdot
		\pr{\text{you flip more than 20 times}}\right) \\
	& = &	\left(\sum_{k=1}^{20} 2^k \cdot \frac{1}{2^k}\right) +
		\left(2^{20} \cdot \frac{1}{2^{20}}\right) \\
	& = &	21
\eeqn

\noindent Under the reasonable assumption that you won't really get
paid more than a million dollars, the expected payoff is a measley
\$21!

\end{document}
