Massachusetts Institute of Technology Lecture ?? 6.042J/18.062J: Mathematics for Computer Science November 7, 1996 Professors Nancy Lynch and Tom Leighton

Notes 18

1 Generating Functions

Definition 1.1 Let < g_{0}, g_{1}, g_{2}, … > be an infinite sequence. The (ordinary) generating function of the sequence is the formal power series

G(x) = ∑_{n=0}^{∞} g_{n}x^{n} (= ∑_{n} g_{n}x^{n} simplified notation) .

Note that x is formal, i.e. symbolic. Do not actually evaluate G(x), just use its structure.

Examples:

G(x) = \frac{1}{1 − x} = ∑_{n} 1 · x^{n} ⟷< 1, 1, 1, … >

G(x) = 1 + x ⟷< 1, 1, 0, 0, 0, … >

G(x) = (1 + x)^{r} = ∑_{n} \binom{r}{n}x^{n} ⟷< \binom{r}{0}, \binom{r}{1}, \binom{r}{2}, …, \binom{r}{r}, 0, 0, 0, … >

< 1, a, a^{2}, a^{3}, … >⟷ ∑_{n} a^{n}x^{n} = \frac{1}{1 − ax}

< 1, −1, 1, −1, 1, −1, … >⟷ \frac{1}{1 + x} (a = −1 above)

< 1, 0, 1, 0, 1, 0, … >⟷ ∑_{n} a^{n}x^{2n} = ∑_{n} a^{n}(x^{2})^{n} = \frac{1}{1 − x^{2}}

Or,

< 1, 0, 1, 0, 1, 0, … >⟷ \frac{1}{2} (\frac{1}{1 − x} + \frac{1}{1 + x}) = \frac{1}{1 − x^{2}}

< \underbrace{0, 0, 0, …, 0}_{k}, 1, 1, 1, … >⟷ x^{k} \frac{1}{1 − x}

< 1, 2, 3, 4, … >⟷ ∑_{n}(n + 1)x^{n} = \frac{d}{dx} ∑_{n} x^{n} = \frac{d}{dx} \frac{1}{1 − x} = \frac{1}{(1 − x)^{2}}

< 1, 4, 9, 16, … >⟷ \frac{1 + x}{(1 − x)^{3}}

We can get the last one from the derivative rule:

< 1 · 1, 2 · 2, 3 · 3, 4 · 4, … >⟷ \frac{d}{dx} < 0, 1, 2, … >= \frac{d}{dx} \frac{x}{(1 − x)^{2}} = \frac{1 + x}{(1 − x)^{3}}

Ordinary Generating Functions (OGF) represent all elements of a sequence as a single formula. Two useful rules are: OGF’s are can be combined in linear fashion.

< f_{0}, f_{1}, f_{2}, … >⟷ F (x) and < g_{0}, g_{1}, g_{2}, … >⟷ G(x)
⟹ < αf_{0} + βg_{0}, αf_{1} + βg_{2}, αf_{2} + βg_{2}, … >⟷ αF (x) + βG(x)

Derivatives of OGF are also OGF.

< f_{0}, f_{1}, f_{2}, … >⟷ F (x) ⟹< f_{1}, 2f_{2}, 3f_{3}, 4f_{4}, … >⟷ F′(x)

Solving recurrenences using OGF’s. We can use OGF to solve many recurrences. For example, the Fibanocci recurrence is

< 0, 1, 1, 2, 3, 5, 8, 11, … >⟷ \frac{x}{1 − x − x^{2}}

Lecture ??: Notes 18 3 Recall, the Fibanocci recurrence is defined as

a_{n} = \{ \begin{matrix}0 & if n = 0 \\ 1 & if n = 1 \\ a_{n−1} + a_{n−2} & if n ≥ 2 \end{matrix}

1. Define sequence and OGF of sequence. < a_{0}, a_{1}, a_{2}, … >

A(x) = ∑_{n} a_{n}x^{n}

2. Multiply both sides of recurrence by x^{n} and sum over valid indices.

∑_{n=2}^{∞} a_{n}x^{n} = ∑_{n=2}^{∞}(a_{n−1} + a_{n−2})x^{n}

3. Express in terms of OGF LHS = A(x) − a_{0} − a_{1}x = A(x) − x, since a_{0} = 0 and a_{1} = 1

RHS = ∑_{n=0}^{∞}(a_{n} + a_{n+1})x^{n+2}
= x^{2} ∑_{n} a_{n}x^{n} + x ∑_{n} a_{n+1}x^{n+1}
= x^{2}A(x) + x(∑_{n} a_{n}x^{n} − a_{0})
= x^{2}A(x) + xA(x)

Therefore,

A(x) − x = x^{2}A(x) + xA(x)

4. Solve for A(x)

A(x)(1 − x − x^{2}) = x ⟹ A(x) = \frac{x}{(1 − x − x^{2})}

5. Solve for a_{n} = coefficient of x^{n} in A(x). (Often use partial fractions if rational functions)

Using partial fraction decompositon, we see

A(x) = \frac{1}{\sqrt{5}} (\frac{1}{1 − α_{1}x} − \frac{1}{1 − α_{2}x})

where α_{1} = \frac{1+\sqrt{5}}{2} and α_{2} = \frac{1−\sqrt{5}}{2}. Coeff of x^{n} in \frac{1}{1−α_{1}x} is α_{1}^{n}. Coeff of x^{n} in \frac{1}{1−α_{2}x} is α_{2}^{n}. Therefore,

a_{n} = \frac{1}{\sqrt{5}} [(\frac{1 + \sqrt{5}}{2})^{n} + (\frac{1 − \sqrt{5}}{2})^{n}] .

2 OGF’s for Combinations

Recall,

< \binom{r}{0}, \binom{r}{1}, \binom{r}{2}, …, \binom{r}{r}, 0, 0, 0, … >⟷ (1 + x)^{r}

Suppose we have r distinguishable objects {a_{1}, …, a_{r}}. What is the OGF of the number of ways to choose objects from {a_{1}}? g_{0} = 1 g_{1} = 1 g_{2} = 0 g_{3} = 0 g_{4} = 0 etc. G(x) = 1 + x What is the OGF of the number of ways to choose objects from {a_{2}}? Also, 1 + x. What is the OGF of the number of ways to choose objects from {a_{1}, a_{2}}? (1 + x)(1 + x) = (1 + x)^{2} = 1 + 2x + x^{2} In (1 + x), the 1 represents not choosing a_{i}, the x represents choosing a_{i}. Thus, we have 1 way to choose 0 elements, 2 ways to choose 1 element, and 1 way to choose two elements. The coefficient of x^{n} is the number of ways to choose n elements.

In general, (1 + x)^{r} is the OGF for the number of ways to choose from {a_{1}, …, a_{r}}.

Theorem 2.1 Let A(x) and B(x) be OGF’s for the number of ways to choose objects from sets A and B, where A ∩ B = ∅. Then, the OGF for the number of ways of choosing objects from A ∪ B is A(x) · B(x).

Proof. Let A(x) = ∑_{n} a_{n}x^{n}, B(x) = ∑_{n} b_{n}x^{n}.

Let C(x) = A(x)B(x) = ∑_{n} c_{n}x^{n}.

What is c_{n}?

c_{n} = ∑_{k=0}^{n} a_{k}b_{n−k}

< c_{0}, c_{1}, … > is the convolution sequence of < a_{0}, a_{1}, … > and < b_{0}, b_{1}, … >.

Now, how many ways is there to choose n objects from A ∪ B? For k = 0, 1, 2, …, n, pick k objects from A and n − k objects from B. Rule of product ⟹ a_{k}b_{n−k} ways for a given k.

Total = ∑_{k=0}^{n} a_{k}b_{n−k} = c_{n}. Fruit example; Consider choosing a bag of n fruits such that

• No. of apples is even

• No. of bananas is a multiple of 5

• No. of oranges is less than or equal to 4

• No. of pears is either 0 or 1.

How many different bags of fruit are there?

For n = 6, we can list the ways. A 6 4 4 2 2 0 0 B 0 0 0 0 0 5 5 O 0 2 1 4 3 1 0 P 0 0 1 0 1 0 1

For arbitrary n, we can compute the generating function for each fruit and then multiply. Apples: A(x) = 1 + x^{2} + x^{4} + … = \frac{1}{1−x^{2}} . Bananas: B(x) = 1 + x^{5} + x^{10} + … = \frac{1}{1−x^{5}} . Oranges: O(x) = 1 + x + x^{2} + x^{3} + x^{4} = \frac{1−x^{5}}{1−x}. Pears: P (x) = 1 + x = \frac{1−x^{2}}{1−x}.

Total OGF:

F (x) = A(x)B(x)O(x)P (x) = \frac{1}{1 − x^{2}} · \frac{1}{1 − x^{5}} · \frac{1 − x^{5}}{1 − x} · \frac{1 − x^{2}}{1 − x} = \frac{1}{(1 − x)^{2}} = ∑_{n}(n + 1)x^{n}

The answer is (n + 1) ways for a bag of n fruits. Another example: A has ∞ red objects. B has ∞ blue objects. Count the number of ways of picking n objects from A and B. OGF for the number of ways of picking red (blue) objects is 1 + x + x^{2} + … = \frac{1}{1−x}. OGF for the number of ways of picking red or blue objects is

\frac{1}{1 − x} · \frac{1}{1 − x} = \frac{1}{1 − x^{2}} = ∑_{n}(n + 1)x^{n}

Example: What is the OGF of the number of ways to choose objects from k types, where repeated types are allowed? The OGF is (1 + x + x^{2} + …)^{k} = (1 − x)^{−k} We can evaulate the coefficients two ways. One way is to use the generalized binomial theorem.

(1 − x)^{−k} = ∑_{n} \binom{−k}{n}(−1)^{n}x^{n}
= ∑_{n} \frac{(−k)(−k − 1) … (−k − n + 1)}{n!}(−1)^{n}x^{n}
= ∑_{n} \frac{(k)(k + 1) … (k + n − 1)}{n!}x^{n}
= ∑_{n} \binom{k + n − 1}{n}x^{n}

Another way is to evaulate the terms in the Taylor series using derivatives. Recall, from Calculus, that the nth term of Taylor series, f_{n}, of the function F (x) around 0 is F^{(n)}(0)/n!. In our case,

F (x) = (1 − x)^{−k}

Thus, f_{0} = F (0)/0! = 1 F′(x) = k(1 − x)^{−(k+1)} ⟹ f_{1} = F′(0)/1! = k F′′(x) = k(k + 1)(1 − x)^{−(k+2)} ⟹ f_{2} = F′(0)/2! = k(k + 1)/2 In general,

F^{(n)}(x) = k(k + 1) … (k + n − 1)(1 − x)^{−(k+n)}
⟹ f_{n} = F′(0)/n! = k(k + 1) … (k + n − 1)/n! = \binom{k+n−1}{n}.