Massachusetts Institute of Technology Handout ?? 6.042J/18.062J: Mathematics for Computer Science December 12, 1995 Professors Nancy Lynch and Tom Leighton

Notes 26

1 Gambler’s Ruin

Today, we are going to talk about random walks and the Gambler’s Ruin problem. This is a nice problem which uses several of the techniques that we have covered throughout the term. The problem is stated as follows. A gambler starts with $N. He bets $1 each time, and wins $1 with probability p, and loses $1 with probability 1 − p. The game ends when either the gambler is broke or has reached $T = $N + $m. We would like to find the probability of a “win” (i.e. reaching $T).

Example 1 Betting on RED in roulette has a winning probability of 18/38 = 9/19. Say we start with $500, which is more likely: quitting after reaching $600 or going broke (bet $1 each time). It turns out we will go broke about 99.997% of the time!!!

We can visualize the Gambler’s Ruin Problem as a random walk. At each step, if we win we go up by 1 (with probability p), otherwise we go down by 1 (with probability 1 − p). If p ≠ 1/2, we say the random walk is biased. Otherwise, we say it is unbiased. If we hit either the top barrier ($T) or the bottom barrier ($0), the game ends.

T 0  1 2 3 N

We are interested in the probability of winning and losing, Pr(W) and Pr(L). It’s not clear whether these add up to 1. It might be possible that we play the game forever. Later, we will show this happens with probability 0.

Definition 1.1 Let W be the event we win (i.e. reach $T before $0). Let D_{t} be the random variable corresponding to the number of dollars held at step t and P_{n} = P r(W |D_{0} = n).

We wish to calculate P_{n}. We could potentially try to look at the entire sample space of W-L sequences and then count up the probabilities of all winning sequences. Instead, we are going to use a recursive approach. We claim

P_{n} = \{ \begin{matrix}0 & if n = 0 \\ 1 & if n = T \\ pP_{n+1} + (1 − p)P_{n−1} & if 0 < n < T \end{matrix}

The first two conditions follow by the definition of a win or lose. The last condition is derived as follows. Let E be the event of a win on the first bet. Then,

P_{n} = P r(W |D_{0} = n)
= P r(W ∧ E|D_{0} = n) + P r(W ∧ Ē|D_{0} = n) Law of Total Probability
= P r(E|D_{0} = n)P r(W |E ∧ D_{0} = n) + P r(Ē|D_{0} = n)P r(W |Ē ∧ D_{0} = n) Cond. Prob.
= pP r(W |D_{1} = n + 1) + (1 − p)P r(W |D_{1} = n − 1)
= pP r(W |D_{0} = n + 1) + (1 − p)P r(W |D_{0} = n − 1)
= pP_{n+1} + (1 − p)P_{n−1}

Note that P r(W |D_{1} = n + 1) = P r(W |D_{0} = n + 1), since we play till one of the termination conditions. Thus, we can view the game as resetting at each step. Rewriting, we have pP_{n+1} − P_{n} + (1 − p)P_{n−1} = 0, which is a linear homogeneous equation with two boundary conditions.

The characteristic equation is pα^{2} − α + (1 − p) = 0.

The roots are: \frac{1−p}{p}, 1. These roots are distinct if p ≠ 1/2.

Let’s assume p ≠ 1/2. We will consider the other case later.

Then,

P_{n} = A (\frac{1 − p}{p})^{n} + B1^{n}.

Using the boundary conditions, P_{0} = 0 and P_{T} = 1, we obtain

A = \frac{1}{(\frac{1−p}{p})^{T} − 1}, B = −A.

Thus,

P_{n} = \frac{(\frac{1−p}{p})n}{(\frac{1−p}{p})^{T} − 1} − \frac{1}{(\frac{1−p}{p})^{T} − 1}

= (\frac{p}{1 − p})^{T −n} \frac{1 − (\frac{p}{1−p})^{n}}{1 − (\frac{p}{1−p})^{T}}

≤ (\frac{p}{1 − p})^{T −n}

Example 2 Let p = 9/19, T = 600, n = 500. Then P r(W) ≤ (.9)^{100} ≤ 0.003%.

Example 3 If we let n = $1, 000, 000, 000 and T = n + 100, then the P r(W) ≤ 0.003%. In other words, we expect to lose $1 trillion before winning $100. The last inequality does not depend on where you start. It’s a good approximation if both T and n are large.

Example 4 The probability of winning decreasese exponentially in the amount that you have to win before quitting: Pr(Go up $1000 before going broke) ≤ (9/10)^{1}000. This is NOT very likely!

Example 5 On the other hand, if you start with $10 and quit when broke or double, the probability of winning isn’t so bad...

P r(W) = (.9)^{10} (\frac{1 − .9^{10}}{1 − .9^{20}} ) = 9%

Bad odds, but not astronomically small.

Remember, we assumed that p ≠ 1/2. Let’s consider the case p = 1/2. Now, we have a double root of 1. Thus,

P_{n} = An + B,

Using the conditions, P_{0} = 0 and P_{T} = 1, we derive the solution as

P_{n} = \frac{n}{T} .

Example 6 Let n = 500, T = 600. P r(W) = 500/600 = 5/6. This is about what we would expect. For example, if T = 2n, then the probability of doubling your money or losing all your money is the same – 1/2.

One issue we’d like to resolve is the apparent difference between the games with a biased and unbiased coin. If p is very close to 1/2, we would expect the probabilities of winning to be similar. However, the probability of winning (with a biased coin) is much worse than an unbiased coin. Intuitively, this seems very strange. The correct intuition is as follows. If the boundaries are far from the starting point (i.e. T and 0 are not to close to n), the amount of time for the random walk to reach the boundary is pretty long. With a fair coin, the random walk will have an equal chance of swinging widely up or down. However, with a biased coin, the walk has a downward drift in it. As time progress, the upward swing needed to reach the upward barrier becomes larger, and the downward swing to hit the lower barrier gets smaller. Thus, if it takes a long time to see a large swing, the downward drift will be the dominant factor.

T 0 N  Time  Drift Biased Game Fair game T 0 N  Time  Drift

Earlier, we claimed the probability of playing forever is 0. For example, the sequence W LW LW LW L... would allow the player to play forever. In fact, there are infinite number of sequences that allow the player to play forever. Nevertheless, the probability of all these sequences is 0. What we really mean is they form a set of measure 0. It is possible that one could play forever, but the likelihood of that is arbitrary close to 0.

Theorem 1.2 For 0 < p < 1, the probability that the game never ends is 0.

Proof. Break the time line into intervals of size T. Let E_{k} be the event that we are still playing after k such intervals. Then we claim that

Pr(E_{k}) ≤ (1 − (1 − p)^{T})^{k} → 0 as k → ∞

Why is this true? Note that

Pr(E_{k}) = Pr(E_{1} ∧ E_{2} ∧ … ∧ E_{k})
= Pr(E_{1}) · Pr(E_{2} | E_{1}) · . . . · Pr(E_{k} | E_{1} ∧ E_{2} ∧ … ∧ E_{k−1})
= Pr(E_{1}) · Pr(E_{2} | E_{1}) · . . . · Pr(E_{k} | E_{k−1})

Now since the start interval is at less than T, and we might lose all bets and go broke, we have that Pr(Ē_{i}E_{i−1}) ≥ (1 − p)^{T}. Hence it follows that Pr(E_{i} | E_{i−1}) ≤ 1 − (1 − p)^{T}, and hence the claim that Pr(E_{k}) ≤ (1 − (1 − p)^{T})^{k} holds. It follows that the probability of playing forever is 0.

So the probability of playing forever is 0. But how long will we actually get to play? If the game is biased and we start with $n, you will be broke in Θ(n) time with probability close to 1. The payoff distribution follows a binomial. As you recall, the tails of a binomial shrink expontentially fast.

Example 7 Let p = 9/19. Then we expect to lose 10 out of 19 games. If we play x times, we expect to win 9x/19 times, and lose 10x/19 times. In this case, the number of wins is binomially distributed. It follows that the net losses are ∼ x/19, and hence the game will almost surely be over for x ∼ 19n.

We‘re now going to prove the famous proverb Quit while you’re ahead”. So let’s return to the fair game with an unbiased coin. We’ll change the game a bit, so we either play forever or go broke (which is to say, we DON‘T quit while ahead). What is the probability that we go broke? It turns out to be 1.

Theorem 1.3 For any initial wealth, n, the probability of going broke is 1.

Proof. For any T > n, we have shown that the probability of reaching T before going broke is \frac{n}{T}, and the probability of neither reaching T or 0 is 0. Therefore, the probability of going broke before reaching T is 1 − \frac{n}{T}.

We now proceed by contradiction. Assume the probability of going broke, P is less than 1. (i.e. P < 1).

Choose T > \frac{n}{1−P}. Therefore,

P ≥ P r(going broke before reaching T)
= 1 − \frac{n}{T}
> 1 − \frac{n}{\frac{n}{1−P}}
= 1 − (1 − P)
= P

This implies P > P – a contradiction. Therefore, P = 1.

So far, we’ve been pessimistic. Our final result will be positive. In the fair game, even though you will go broke with probability 1, your expected playing time is infinite. Even if you start with $1.

Theorem 1.4 If n = 1 and p = 1/2, expected time to go broke is infinite.

Proof. Let S be the number of step to go broke. Note that S must be odd. After an even number of steps, the final wealth must be 1+(even). So, S is odd.

Claim: For t odd, P r(S = t) = \frac{2}{t+1} \binom{t−1}{\frac{t−1}{2}}2^{−t}.

Using Stirling’s approximation, we get that the probability of going broke at t is Pr(S = t) = Θ(t^{−3/2}). Therefore,

P r(S > m) = ∑_{t odd,t=m+1}^{∞} Θ(t^{−3/2}) = Θ(1/\sqrt{m}).

Finally,

Ex(S) = ∑_{m=0}^{∞} P r(S > m) = ∑_{m=0}^{∞} Θ(1/\sqrt{m}).

This above sum is unbounded and approaches infinity. Thus, the expected length of the game is infinite.