\documentstyle[12pt,twoside,amstex,eepic]{article}
\input{macros-course}

\parskip1.5ex
\parindent0pt

\begin{document}
\handout{notes26}{Notes 26}{December 12, 1995}

\section{Gambler's Ruin}
Today, we are going to talk about random walks and the Gambler's Ruin
problem.  This is a nice problem which uses several of the techniques
that we have covered throughout the term. The problem is stated as
follows.  A gambler starts with \$N.  He bets \$1 each time, and wins
\$1 with probability $p$, and loses \$1 with probability $1-p$.  The
game ends when either the gambler is broke or has reached \$T = \$N +
\$m. We would like to find the probability of a ``win'' (i.e. reaching
\$T).
\begin{example}
  Betting on RED in roulette has a winning probability of
  $18/38=9/19$.  Say we start with \$500, which is more likely:
  quitting after reaching \$600 or going broke (bet \$1 each time).
  It turns out we will go broke about 99.997\% of the time!!!
\end{example}

We can visualize the Gambler's Ruin Problem as a random walk.  At each
step, if we win we go up by 1 (with probability $p$), otherwise we go
down by 1 (with probability $1-p$).  If $p \neq 1/2$, we say the
random walk is biased. Otherwise, we say it is unbiased.  If we hit
either the top barrier (\$T) or the bottom barrier (\$0), the game
ends.

\setlength{\unitlength}{0.0125in}
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(300,175)(0,-10)
\path(40,160)(40,0)
\path(20,40)(300,40)
\dottedline{5}(20,140)(300,140)
\path(40,80)(60,100)(80,80)
        (100,100)(120,120)(140,100)
        (160,120)(180,100)(200,80)
        (220,60)(240,80)(260,100)
        (280,120)(300,140)
\put(0,140){\makebox(0,0)[lb]{\smash{{{\SetFigFont{12}{14.4}{rm}T}}}}}
\put(0,40){\makebox(0,0)[lb]{\smash{{{\SetFigFont{12}{14.4}{rm}0}}}}}
\put(60,20){\makebox(0,0)[lb]{\smash{{{\SetFigFont{12}{14.4}{rm}1}}}}}
\put(80,20){\makebox(0,0)[lb]{\smash{{{\SetFigFont{12}{14.4}{rm}2}}}}}
\put(100,20){\makebox(0,0)[lb]{\smash{{{\SetFigFont{12}{14.4}{rm}3}}}}}
\put(0,80){\makebox(0,0)[lb]{\smash{{{\SetFigFont{12}{14.4}{rm}N}}}}}
\end{picture}


We are interested in the probability of winning and losing, $\Pr(W)$
and $\Pr(L)$.  It's not clear whether these add up to 1.  It might be
possible that we play the game forever.  Later, we will show this
happens with probability 0.

\begin{defin}
Let $W$ be the event we win (i.e. reach \$T before \$0).
Let $D_t$ be the random variable corresponding to the number
of dollars held at step $t$ and $P_n= Pr(W|D_0=n)$.
\end{defin}

We wish to calculate $P_n$. We could potentially try to look at the
entire sample space of W-L sequences and then count up the
probabilities of all winning sequences. Instead, we are going to use a
recursive approach.  We claim
\[
P_n = \left\{ \begin{array}{ll}
                0 & \mbox{if $n=0$} \\
                1 & \mbox{if $n=T$} \\
                pP_{n+1}+(1-p)P_{n-1} & \mbox{if $0<n<T$}
              \end{array}
        \right.
\]

The first two conditions follow by the definition of a win or lose.
The last condition is derived as follows.  Let $E$ be the event of
a win on the first bet.  Then, 

\begin{eqnarray*}
P_n &=&  Pr(W|D_0=n) \\
&=&  Pr(W\wedge E|D_0=n) + Pr(W\wedge \bar{E}|D_0=n) \mbox{ Law of Total Probability} \\
&=&  Pr(E|D_0=n) Pr(W|E\wedge D_0=n) +  Pr(\bar{E}|D_0=n) Pr(W|\bar{E}\wedge D_0=n) \mbox{ Cond. Prob.}\\
&=&  pPr(W|D_1=n+1)+(1-p)Pr(W|D_1=n-1) \\
&=&  pPr(W|D_0=n+1)+(1-p)Pr(W|D_0=n-1) \\
&=&  pP_{n+1}+(1-p)P_{n-1} \\
\end{eqnarray*}

Note that $Pr(W|D_1=n+1)=Pr(W|D_0=n+1)$, since we play till one of the
termination conditions.  Thus, we can view the game as resetting at
each step.  Rewriting, we have $pP_{n+1}-P_n +(1-p)P_{n-1}=0$, which
is a linear homogeneous equation with two boundary conditions.

The characteristic equation is $p\alpha^2 - \alpha +(1-p)=0$.

The roots are:  $\frac{1-p}{p}$, 1.   These roots are distinct if $p\neq 1/2$.

Let's assume $p\neq 1/2$.  We will consider the other case later.

Then, 
\[
P_n = A \left(\frac{1-p}{p}\right)^n + B 1^n.
\]

Using the boundary conditions, $P_0=0$ and $P_T=1$, we obtain
\[
A=\frac{1}{\left(\frac{1-p}{p}\right)^T -1}, B= -A.
\]

Thus, 
\begin{eqnarray*}
P_n& =& 
\frac{\left(\frac{1-p}{p}\right)^n}{\left(\frac{1-p}{p}\right)^T -1}
- \frac{1}{\left(\frac{1-p}{p}\right)^T -1}\\
&=& \left(\frac{p}{1-p}\right)^{T-n} 
\frac{1- \left(\frac{p}{1-p}\right)^n}{1- \left(\frac{p}{1-p}\right)^T} \\
&\leq& \left(\frac{p}{1-p}\right)^{T-n}
\end{eqnarray*}


\begin{example}
Let $p=9/19$, $T=600$, $n=500$. Then $Pr(W) \leq (.9)^{100} \leq
0.003\%$.
\end{example}

\begin{example}
If we let $n=\$1,000,000,000$ and $T=n+100$, then the $Pr(W)\leq
0.003\%$.  In other words, we expect to lose \$1 trillion before
winning \$100.  The last inequality does not depend on where you start.
It's a good approximation if both $T$ and $n$ are large.
\end{example}

\begin{example}
The probability of winning decreasese exponentially in the amount that
you have to win before quitting:\\
$\Pr(\text{Go up \$1000 before going broke}) \leq (9/10)^1000$. This
is NOT very likely!
\end{example}

\begin{example}
On the other hand, if you start with \$10 and quit when broke or
double, the probability of winning isn't so bad...\\

$$Pr(W) = (.9)^{10} \left(\frac{1-.9^{10}}{1-.9^{20}}\right) = 9\%$$
Bad odds, but not astronomically small.
\end{example}

Remember, we assumed that $p\neq 1/2$.  Let's consider the case
$p=1/2$.  Now, we have a double root of 1.  Thus, 
\[
P_n=An+B,
\]

Using the conditions, $P_0 = 0$ and $P_T=1$, we derive the solution as
\[
P_n = \frac{n}{T}.
\]

\begin{example}
Let  $n=500$, $T=600$.  $Pr(W) = 500/600 = 5/6$.
This is about what we would expect.  For example, if $T=2n$, then
the probability of doubling your money or losing all your money is the
same -- 1/2.
\end{example}

One issue we'd like to resolve is the apparent difference between the
games with a biased and unbiased coin.  If $p$ is very close to 1/2,
we would expect the probabilities of winning to be similar.  However,
the probability of winning (with a biased coin) is much worse than an
unbiased coin.  Intuitively, this seems very strange.  The correct
intuition is as follows.  If the boundaries are far from the starting
point (i.e. $T$ and $0$ are not to close to $n$), the amount of time
for the random walk to reach the boundary is pretty long.  With a fair
coin, the random walk will have an equal chance of swinging widely up
or down.  However, with a biased coin, the walk has a downward drift
in it.  As time progress, the upward swing needed to reach the upward
barrier becomes larger, and the downward swing to hit the lower
barrier gets smaller.  Thus, if it takes a long time to see a large
swing, the downward drift will be the dominant factor.

\setlength{\unitlength}{0.0125in}
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(397,121)(0,-10)
\path(244,106)(244,12)
\dottedline{5}(232,94)(397,94)
\path(232,23)(397,23)
\path(244,59)(256,71)(267,59)
        (279,47)(291,35)(302,47)
        (314,35)(326,47)(337,59)
        (349,47)(362,35)(373,47)
        (385,35)(397,23)
\dottedline{5}(244,59)(397,23)
\put(221,94){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}T}}}}}
\put(221,35){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}0}}}}}
\put(221,59){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}N}}}}}
\put(267,12){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}Time}}}}}
\put(397,23){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}Drift}}}}}
\put(279,0){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}Biased Game}}}}}
\path(23,106)(23,12)
\dottedline{5}(12,94)(176,94)
\path(23,59)(35,71)(47,59)
        (58,71)(71,83)(83,71)
        (94,83)(106,71)(118,59)
        (129,47)(141,59)(153,71)
        (164,83)(176,94)
\path(12,23)(176,23)
\dottedline{5}(23,59)(176,59)
\put(58,0){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}Fair game}}}}}
\put(0,94){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}T}}}}}
\put(0,35){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}0}}}}}
\put(0,59){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}N}}}}}
\put(47,12){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}Time}}}}}
\put(176,59){\makebox(0,0)[lb]{\smash{{{\SetFigFont{7}{8.4}{rm}Drift}}}}}
\end{picture}


Earlier, we claimed the probability of playing forever is 0.
For example, the sequence $WLWLWLWL ...$ would allow
the player to play forever.  In fact, there are infinite number of sequences
that allow the player to play forever.  Nevertheless, the probability
of all these sequences is 0.  What we really mean is they form a set
of measure 0.  It is possible that one could play forever, but the
likelihood of that is arbitrary close to 0.


\begin{theorem}
For $0<p<1$, the probability that the game never ends is 0.
\end{theorem}

\begin{proof}
Break the time line into intervals of size $T$. Let $E_k$ be the event
that we are still playing after $k$ such intervals. Then we claim
that\\

$$\Pr(E_k) \leq (1-(1-p)^T)^k \rightarrow 0 \,\text{as}\, k \rightarrow
\infty$$

Why is this true? Note that\\

\begin{align*}
\Pr(E_k) &= \Pr(E_1 \wedge E_2 \wedge \ldots \wedge E_k)\\
&= \Pr(E_1)\cdot \Pr(E_2 \mid E_1)\cdot \ldots \cdot \Pr(E_k \mid E_1
\wedge E_2 \wedge \ldots \wedge E_{k-1})\\
&= \Pr(E_1)\cdot \Pr(E_2 \mid E_1)\cdot \ldots \cdot \Pr(E_k \mid
E_{k-1})
\end{align*}

Now since the start interval is at less than $T$, and we {\em might}
lose all bets and go broke, we have that $\Pr(\bar{E_i}\id E_{i-1})
\geq (1-p)^T$.  Hence it follows that $\Pr(E_i \mid E_{i-1}) \leq
1-(1-p)^T$, and hence the claim that $\Pr(E_k) \leq (1-(1-p)^T)^k$
holds. It follows that the probability of playing forever is $0$.
\end{proof}

So the probability of playing forever is $0$. But how long will we
actually get to play? If the game is biased and we start with \$n, you
will be broke in $\Theta(n)$ time with probability close to 1.  The
payoff distribution follows a binomial.  As you recall, the tails of a
binomial shrink expontentially fast.

\begin{example}
  Let $p=9/19$. Then we expect to lose $10$ out of $19$ games. If we
  play $x$ times, we expect to win $9x/19$ times, and lose $10x/19$
  times. In this case, the number of wins is binomially distributed.
  It follows that the net losses are $\sim x/19$, and hence the game
  will almost surely be over for $x \sim 19n$.
\end{example}


We`re now going to prove the famous proverb ~Quit while you're
ahead''.  So let's return to the fair game with an unbiased coin.
We'll change the game a bit, so we either play forever or go broke
(which is to say, we DON`T quit while ahead).  What is the probability
that we go broke?  It turns out to be 1.

\begin{theorem}
For any initial wealth, $n$, the probability of going broke
is 1.
\end{theorem}

\begin{proof}
For any $T>n$, we have shown that the probability of reaching $T$ 
before going broke is $\frac{n}{T}$, and the probability of neither
reaching $T$ or 0 is 0.  Therefore, the probability of going broke
before reaching $T$ is
$1-\frac{n}{T}$.

We now proceed by contradiction.  Assume the probability of
going broke, $P$ is less than 1.  (i.e. $P<1$).

Choose $T> \frac{n}{1-P}$.  Therefore, 
\begin{eqnarray*}
P & \geq & Pr(\mbox{going broke before reaching $T$})\\
&=& 1-\frac{n}{T}\\
&>&1-\frac{n}{\frac{n}{1-P}}\\
&=&1-(1-P)\\
&=&P
\end{eqnarray*}

This implies $P>P$ -- a contradiction.
Therefore, $P=1$.
\end{proof}

So far, we've been pessimistic.  Our final result will be positive.
In the fair game, even though you will go broke with probability 1, 
your expected playing time is infinite.  Even if you start with \$1.

\begin{theorem}
If $n=1$ and $p=1/2$, expected time to go broke is infinite.
\end{theorem}

\begin{proof}
Let $S$ be the number of step to go broke.
Note that $S$ must be odd.  After an even number of steps,
the final wealth must be 1+(even).  So, $S$ is odd.

Claim:  For $t$ odd, $Pr(S=t) = \frac{2}{t+1} {t-1 \choose {\frac{t-1}{2}}}
2^{-t}$.

Using Stirling's approximation, we get that the probability of going
broke at $t$ is $\Pr(S=t)=\Theta(t^{-3/2})$.
Therefore, 
\[
Pr(S>m) = 
\sum_{t\mbox{ odd,}t=m+1}^\infty \Theta(t^{-3/2}) = \Theta(1/\sqrt{m}).
\]

Finally,
\[
Ex(S) = \sum_{m=0}^\infty Pr(S>m) = \sum_{m=0}^\infty \Theta(1/\sqrt{m}).
\]

This above sum is unbounded and approaches infinity.  Thus, the
expected length of the game is infinite.
\end{proof}

\end{document}

