Massachusetts Institute of Technology Lecture ?? 6.042J/18.062J: Mathematics for Computer Science October 24, 1996 Professors Nancy Lynch and Tom Leighton

Notes 14

1 Linear Recurrences

We begin with an example. Graduate Student Job Prospects Problem: Let’s assume that due to budgetary constraints, there are at most N faculty positions available. Every year, each professor graduates one student who become a professor upon graduation. Except, first year professor are too busy and don’t graduate any students. To make matters worse, everyone lives forever and no one retires. The question that graduate students ask is: will there be any positions when I graduate? That is, when will all the jobs be filled. We also impose the boundary condition that in year 1, there is one professor. To solve this problem, we will set up a recurrence. Let f(n) be the number of professors in year n. The first few values are, f(0) = 0 f(1) = 1 (new hire) f(2) = 1 (no students yet) f(3) = 2 (one new prof) f(4) = 3 (2 old, 1 new) f(5) = 5 (3 old, 2 new) For n ≥ 2, f(n) = f(n − 1) + f(n − 2). The Fibanocci recurrence. We can now define linear recurrence.

Definition 1.1 A recurrence is linear if it is of the form

f(n) = a_{1}f(n − 1) + a_{2}f(n − 2) + … a_{d}f(n − d) = ∑_{i=1}^{d} a_{i}f(n − i)

for fixed constants a_{1}, a_{2}, a_{3}, …, a_{d}, and d.

2 Lecture ??: Notes 14 Definition 1.2 The order of the recurrence is d. (e.g., the Fibanocci recurrence has order 2)

2 Solving the Recurrence

Let’s try the guess and verify method. Guess: f(n) = α^{n} for constant α. Verify:

f(n) = f(n − 1) + f(n − 2)
⟹  α^{n} = α^{n−1} + α^{n−2}
⟹  α^{2} = α + 1
⟹  α^{2} − α − 1 = 0
⟹  α = \frac{1±\sqrt{5}}{2}

So, we have two possible solutions: f(n) = α_{1}^{n} and f(n) = α_{2}^{n} where α_{1}, α_{2} = \frac{1±\sqrt{5}}{2}. In fact, any linear combination is a solution. Theorem 2.1 If f(n) = α_{1}^{n} and f(n) = α_{2}^{n} are two solutions to a linear recurrence (without boundary conditions), then f(n) = c_{1}α_{1}^{n} + c_{2}α_{2}^{n} is also a solution for any constant c_{1} and c_{2}. Proof.

α_{1}^{n} is a solution ⟹ α_{1}^{n} = ∑_{i=1}^{d} a_{i}α_{1}^{n−i}

α_{2}^{n} is a solution ⟹ α_{2}^{n} = ∑_{i=1}^{d} a_{i}α_{2}^{n−i}

Thus,

c_{1}α_{1}^{n} + c_{2}α_{2}^{n} = ∑_{i=1}^{d} a_{i}(c_{1}α_{1}^{n−i} + c_{2}α_{2}^{n−i})

So,

f(n) = ∑_{i=1}^{d} a_{i}f(n − i)

Thus, we know the solution to the Fibanacci recurrence is

f(n) = c_{1} (\frac{1 + \sqrt{5}}{2})^{n} + c_{2} (\frac{1 − \sqrt{5}}{2})^{n}

To determine the constants, we use the boundary conditions.

f(0) = 0 = c_{1} + c_{2}

f(1) = 1 = c_{1} (\frac{1 + \sqrt{5}}{2}) + c_{2} (\frac{1 − \sqrt{5}}{2})

Thus, c_{1} = 1/\sqrt{5} and c_{2} = −1/\sqrt{5}. Surprisingly, the above formula gives the Fibanocci numbers, which is a non-obvious result. In fact, it isn’t even clear the above number is always an integer.

To answer our original question, we want the smallest n such that f(n) ≥ N. Observe that (\frac{1−\sqrt{5}}{2}) < 1, so any powers of it are also less than 1. For large values of n, this term will becomes very close to zero, and thus the first term converges to the nth Fibanocci number. Even for small values of n, the first term will give us the answer to within 1.

\frac{1}{\sqrt{5}} (\frac{1 + \sqrt{5}}{2})^{n} ≥ N ⟹ n ≥ \frac{log(\sqrt{5}N)}{log (\frac{1+\sqrt{5}}{2})} = Θ(log N)

For numerical values of N, we can give accurate answers. For N = 20, 000, we have n = 21. In general, we are most interested in asymptotics. In that case, the jobs will be filled in Θ(log N) years.

3 Solving General Recurrences

For an arbitrary dth order linear recurrence,

f(n) = ∑_{i=1}^{d} a_{i}f(n − i)

Guess: f(n) = α^{n} for some α Verify:

f(n) = ∑_{i=1}^{d} a_{i}f(n − i)
⟹  α^{n} = ∑_{i=1}^{d} a_{i}α^{n−i}
⟹  α^{d} = ∑_{i=1}^{d} a_{i}α^{d−i}
⟹ α^{d} − a_{1}α^{d−1} − a_{2}α^{d−2} − … a_{d} = 0

This last equation is called the characteristic equation. In general, this equation will have d roots. If all the roots are distinct, the solution will be

f(n) = c_{1}α_{1}^{n} + c_{2}α_{2}^{n} + … + c_{d}α_{d}^{n}

We determine the c_{i} by using the boundary conditions. We need d equations to find a unique solution.

If the roots are repeated, we need to modify our solutions. If a root, α, is repeated r times, then the solutions look like

α^{n}, nα^{n}, n^{2}α^{n}, …, n^{r−1}.

We will verify this for the case r = 2. (i.e. nα^{n} is a solution) Note that if α is a double root of the characteristic equation, then let g(x) = (x^{d} − a_{1}x^{d−1} − a_{2}x^{d−2} − … a_{d})x^{n−d}, then

g(x) = (x − α)^{2}h(x)
⟹ g′(x) = 2(x − α)h(x) + (x − α)^{2}h′(x)

Thus, g′(α) = 0. Therefore,

nα^{n−1} − ∑_{i=1}^{d} a_{i}(n − i)α^{n−i−1} = 0
⟹  nα^{n} = ∑_{i=1}^{d} a_{i}(n − i)α^{n−i}

which shows that nα^{n} is a solution.

Example: We have a plant that lives forever. It produces one offspring the first year of its life and nothing in the future. If we start with one new plant in year 1, how many plants will we have in year n?

Recurrence:

f(n) = f(n − 1) + (f(n − 1) + f(n − 2)) = 2f(n − 1) − f(n − 2)

At year n, we have the plants from last year plus the one new plant for each first year plant.

Boundary conditions: f(0) = 0, f(1) = 1.

Characteristic equation: α^{2} − 2α + 1 = 0. Roots: ±1. Solution: c_{1} + c_{2}n Using boundary conditions, we get f(n) = n. Recurrences of the form

f(n) − ∑_{i=1}^{d} a_{i}f(n − i) = 0

are called homogeneous recurrences. We next study inhomogeneous recurrences.

4 Inhomogeneous Recurrences

Inhomogeneous recurrences look like

f(n) − ∑_{i=1}^{d} a_{i}f(n − i) = g(n)

The general method is to do the following:

1. Solve the homogeneous solution by setting g(n) = 0.

2. Find a particular solution.

3. Add the two together and use the boundary conditions to determine the coefficients of the constant factors.

For those who have taken 18.03, this is similiar to solving differential equations. Example: f(n) = 4f(n − 1) + 3^{n}; f(1) = 1 Homogeneous solution: f(n) − 4f(n − 1) = 0. α = 4. f(n) = c4^{n} Particular solution: Guess f(n) = k3^{n}

k3^{n} = 4k3^{n−1} + 3^{n}
3k = 4k + 3
k = −3

Particular solution: f(n) = −3^{n+1} Combine solutions: f(n) = c4^{n} − 3^{n+1}. Boundary conditions imply f(n) = \frac{5}{2}4^{n} − 3^{n+1}.

Another example: f(n) = 4f(n − 1) − 4f(n − 2) + n; f(0) = 4, f(1) = 7 Homogeneous solution: f(n) − 4f(n − 1) + 4f(n − 2) = 0. α = 2, 2 (double root). f(n) = c_{1}2^{n} + c_{2}n2^{n} Particular solution: Guess f(n) = an + b

an + b = 4(a(n − 1) + b) − 4(a(n − 2) + b) + n
an + b = n + 4a

Thus, a = 1, b = 4. Particular solution: f(n) = n + 4 Combine solutions: f(n) = c_{1}2^{n} + c_{2}n2^{n} + n + 4. Boundary conditions imply f(n) = n2^{n} + n + 4. When guessing a particular solution, it makes sense to guess a solution that is the same type as g(n). If g(n) is an r-degree polynomial, guess an r-degree polynomial (i.e., ax^{r} + bx^{r−1} + … + d). If g(n) is an exponential, guess an exponential(e.g., if g(n) = k^{n}, guess ck^{n}).