Massachusetts Institute of Technology Lecture 2 6.042J/18.062J: Mathematics for Computer Science September 10, 1996 Professors Nancy Lynch and Tom Leighton

Notes 2

Nearly all of today, and next lecture, are devoted to one of the most important proof techniques for discrete structures: mathematical induction. You’ve already seen this method in high school, but we want you to get really good at using it. It’s really practical.

It’s used for proving properties of natural numbers, and of objects defined with a recursive structure (trees, etc.). It’s also the basis of much of the work on formal verification of programs and algorithms.

Before starting this, two brief remarks, about formal axiom systems and about mathematical definitions.

4 Formal Axiom Systems

In recitation, you saw Peano’s axioms, a commonly-accepted basis for number theory.

Peano Axioms for ℕ:

1. 0 ∈ ℕ

2. ∀x ∈ ℕ, x′ ∈ ℕ

3. ∀x, y ∈ ℕ, x′ = y′ ⟹ x = y (no two numbers have same successor)

4. ¬∃x ∈ ℕ such that 0 = x′ (equivalently, ∀x ∈ ℕ, x′ ≠ 0)

5. (Induction) ∀S ⊆ N, if 0 ∈ S and (x ∈ S ⟹ x′ ∈ S), then S = ℕ.

Using these axioms, rules of inference, additional definitions, can develop all of number theory, very formally.

In practice, people do not write out real proofs about interesting math objects in such a formal, step-by-step manner. (Unless they are using a computer-aided theorem-prover, which really does work step-by-step.)

Rather, work intuitively, skipping lots of steps. Proof sketch.

But it is very important to work correctly, so that all the missing steps could really be filled in. It can be very hard to tell (without training) if a proof sketch really is valid. This course is designed to provide this training.

5 Definitions

When working in a logical proof system, we start with some basic concepts (for Peano: natural numbers, 0, successor), use in axioms, theorems. Then define other concepts in terms of these, build...

E.g., for natural numbers, define addition in term of successor. Can continue, defining multiplication in terms of addition, then subtraction and division as inverses, etc.

Definitions are not strictly necessary: we could always write out the defined object in terms of basic notions. But this would get messy fast. Definitions are used to establish convenient shorthand.

5.1 What Definitions to Make?

Definitions call attention to something interesting that we expect to encounter frequently. Some typical examples from number theory:

• A factor of a natural number n is a positive integer that divides n without remainder.

• A prime number is a number ≥ 2 whose only factors are 1 and itself.

These are very useful definitions. Save us lots of typing:

Ex: Expanding the second: “A number ≥ 2 for which no number other than itself and 1 divides into it without remainder.”

Prime numbers arise in many situations, e.g., cryptography (RSA), so it’s nice to have a short name for them.

There are many interesting theorems about prime numbers, e.g., Fundamental Theorem of Arithmetic says that all positive integers can be factored into primes, in a unique way.

In contrast, the following would not be a useful definition:

• A K-number is one that is divisible by 5 and by 7, but not by 13 or 43.

This definition also provides a shorthand, but it’s not too useful. Probably won’t want it very often. And there aren’t many interesting results that depend on it.

5.2 Well-Definedness

It is important that a definition really define a concept, unambiguously. Often express this by saying that the concept be well-defined by the definition.

Ex: Addition is uniquely defined by the recurrences given earlier – only one function can satisfy the equations.

Some examples of bad definitions – in which concepts aren’t well-defined:

• f(n) is the largest rational number (strictly) less than n

Sometimes there is no such number.

How to prove this?

Construct a larger one by \frac{n+f(n)}{2}.

• \sqrt{x} is the complex number y such that y^{2} = x.

\sqrt{x} is not well defined by this definition, because it can take on two possible values. Pretending that it is well-defined and proceeding with usual manipulations leads to fallacies like:

1 = \sqrt{1}
= \sqrt{(−1)(−1)}
= \sqrt{(−1)}\sqrt{(−1)}
= −1

5.3 Recursive Definitions

A recursive definition says how to build something from a simpler version of the same thing.

Two parts:

• Base case(s) that don’t depend on anything else

• Combination cases that depend on simpler cases

Note the relationship to a recursive algorithm. Examples:

• Addition, multiplication:

– We already saw recursive definition of addition.

– Do similar for multiplication (in terms of addition):

n × 1 = n

n × k′ = n × k + n

• The Factorial function

(Writing n + 1 for n′.)

– 1! = 1

– (n + 1)! = (n + 1)n! for n ≥ 1.

• The Fibonacci numbers:

– f_{0} = 0

– f_{1} = 1

– f_{n+1} = f_{n} + f_{n−1} Note: 2 base cases, since combination relies on previous two values. What is f_{4}? Well, f_{2} = f_{0} + f_{1} = 1, f_{3} = f_{1} + f_{2} = 2, so f_{4} = 3.

• A well formed arithmetic formula is – A number

– (f + g), (f − g), (f ∗ g), or (f/g) where f, g are well-formed arithmetic formulae.

• A binary tree is one of – A node with no children.

– A node with a left child that is a binary tree, and no right child.

– A node with a right child that is a binary tree, and no left child.

– A node with left and right binary-tree children.

Can also have simultaneous recursive definitions:

• Define two functions f and g from ℕ to ℕ, recursively, by: – f(0) = 1

– g(0) = 1

– f(n + 1) = f(n) + g(n), n ≥ 0

– g(n + 1) = f(n) × g(n), n ≥ 0

6 Proofs By Induction

6.1 What They Are

Induction is a method used to prove that some property holds for all natural numbers, or more generally, for all elements of some set defined by base elements plus generation rules. (E.g., wff’s, binary trees.)

For starters, we’ll consider natural numbers. Simple induction is expressed by Peano’s Induction Axiom:

• (Induction) ∀S ⊆ N, if 0 ∈ S and (x ∈ S ⟹ x′ ∈ S), then S = ℕ.

Or, restating in terms of predicates instead of sets (equivalent), (and writing n + 1 instead of n′):

Let P (n) be a predicate, and suppose

• P (0)

• (∀n) (P (n) ⟹ P (n + 1)).

Then P (n) is true for all natural numbers n.

That is, to prove a property P for all natural numbers, it’s enough to prove it for 0, then to show that truth for any n implies truth for n + 1. Why does this say it’s true for all of ℕ? Well, it’s true for 0 by assumption, then for 1 because it’s true for 0, then for 2 because it’s true for 1, etc.

Actually, we don’t prove that this is a valid rule – we assume it as an axiom. There’s no way to prove it from the other axioms.

Think of dominoes: Want to say that all the dominoes in a row fall down. Assume we push the first one (domino 0) down. Then suppose that any domino that falls causes the next to fall. Then all fall.

An inductive proof is not a fallacious circular proof. We do not prove P (n) ⟹ P (n).

Simple variant: In order to prove P (n) for all n ≥ 1, just prove

• P (1)

• (∀n ≥ 1) (P (n) ⟹ P (n + 1)).

Same for any k.

In most of the rest of the class, we give examples of inductive proofs.

Although we’ve expressed induction as an axiom, we are not going to be so formal here in our reasoning about ℕ. Besides the simple concepts and facts mentioned in the Peano axioms, we are going to use other operations like +, ×, etc., relations like <, and lots of basic properties from elementary school arithmetic.

We will get formal again (briefly) next lecture, in order to consider the relationship between the Peano induction axiom and some alternative induction axioms.

Vital: write down P exactly!

6.2 Simple Number Properties

Simple induction. Just numbers. No need to strengthen hypothesis.

Example 1 Prove for all n ∈ ℕ, n^{3} − n is divisible by 3.

Let P (n) be the predicate n^{3} − n is divisible by 3.

P (0): 0 is divisible by 3. Yes. 0 is divisible by everything, by convention.

Assume P (n), prove P (n + 1): P (n) says n^{3} − n is divisible by 3. P (n + 1) says (n + 1)^{3} − (n + 1) is divisible by 3. Now, (n + 1)^{3} − (n + 1) = (n^{3} + 3n^{2} + 3n + 1) − (n + 1) = (n^{3} − n) + 3(n^{2} + n).

First term divisible by 3, by inductive hypothesis. Second term, obviously. Therefore, sum.

Example 2 Show 2^{n} ≥ n^{2} if n ≥ 4.

P (n) = … BASE: 2^{4} = 4^{2}. INDUCTIVE STEP: Assume 2^{n} ≥ n^{2} and n ≥ 4.

Show 2^{n+1} ≥ (n + 1)^{2}. 2^{n+1} = 2 × 2^{n} ≥ 2 × n^{2} by inductive hypothesis. Need to show 2 × n^{2} ≥ (n + 1)^{2}. Or, 2n^{2} ≥ n^{2} + 2n + 1, or n^{2} ≥ 2n + 1.

So, reduces to showing n^{2} ≥ 2n + 1 for n ≥ 4. How? Can do with another induction: Q(4) true. Have Q(n), n^{2} ≥ 2n + 1. Q(n + 1) says (n + 1)^{2} ≥ 2(n + 1) + 1 = 2n + 3. (n + 1)^{2} = n^{2} + 2n + 1 ≥ 2n + 1 + 2n + 1 by inductive hypothesis = 4n + 2. This is ≥ 2n + 3 because n ≥ 4, as needed.

The following shows how inductive proofs can fit nicely with recursive definitions:

Example 3 Show n! ≥ 4^{n−2} for all n ≥ 2 BASE CASES: 2 and 3, just check. (Technically, only 2 is a base – the other is a trivial inductive step.) INDUCTIVE: Know n! ≥ 4^{n−2}. Want (n + 1)! ≥ 4^{n−1}. LHS multiplies by at least 4, RHS by just 4, so preserved.

An example for two functions defined together, by recursion:

Example 4 The functions f and g above. Prove the following:

1. If n ≥ 2 then f(n) is odd.

2. If n ≥ 2 then g(n) is even.

Can’t prove separately – at least, not the first. So prove the two together by induction. P (n) = conjunction of the two statements. For n = 2, show by explicit calculation (3, 2). Inductive step: Assume f(n) odd and g(n) even. Then sum is odd and product is even, so this is true for n + 1 also.

6.3 Sums of Sequences

An important use of induction is to verify closed-form expressions for sums of sequences.

Theorem 6.1 For any n ≥ 0, 1 + 2 + 3 + ⋯ + n = n(n + 1)/2

Meaning of LHS? Observe the pattern, fill in the dots... Special cases: For n = 1, sum just has one term 1. For n = 0, use default def: Add up no terms, sum of no terms is always 0.

Instead of ⋯ notation, often write in Σ notation: Σ_{i=1}^{n}i

In general, write Σ_{i=1}^{n}f(i) for the sum f(1) + f(2) + ⋯ + f(n). Defined recursively: Σ_{i=1}^{0}f(i) = 0 Σ_{i=1}^{n+1}f(i) = Σ_{i=1}^{n}f(i) + f(n + 1).

Extend to other lower bounds besides 1.

Proof. Let P (n) be the proposition that the sum of the first n positive integers is n(n+1)/2. BASIS STEP: P (0) is true since 0 = 0(0 + 1)/2. INDUCTIVE STEP: Assume that P (n) holds, i.e.,

1 + 2 + 3 + ⋯ + n = n(n + 1)/2.

Add n + 1 to both sides to get that

1 + 2 + 3 + ⋯ + n + (n + 1) = n(n + 1)/2 + (n + 1) = (n + 1)(n + 2)/2.

Hence P (n + 1) is true. This completes the inductive step and the proof.

Can also be proved without induction. How? Another summation example:

Theorem 6.2 For any n ≥ 0, Σ_{i=1}^{n}i^{3} = (Σ_{i=1}^{n}i)^{2}.

Proof. Let P (n) be the proposition for n. By induction. BASIS STEP: P (0) – both are 0. INDUCTIVE STEP: Assume P (n), i.e.,

Σ_{i=1}^{n}i^{3} = (Σ_{i=1}^{n}i)^{2}.

For P (n + 1), we must show:

Σ_{i=1}^{n+1}i^{3} = (Σ_{i=1}^{n+1}i)^{2}.

LHS is just

Σ_{i=1}^{n}i^{3} + (n + 1)^{3},

which by inductive hypothesis is equal to:

(Σ_{i=1}^{n}i)^{2} + (n + 1)^{3}.

RHS is just

(Σ_{i=1}^{n}i + (n + 1))^{2}.

which equals

(Σ_{i=1}^{n}i)^{2} + 2(n + 1)(Σ_{i=1}^{n}i) + (n + 1)^{2}.

Are these two equal? The first terms drop out, so it remains to show:

(n + 1)^{3} = 2(n + 1)(Σ_{i=1}^{n}i) + (n + 1)^{2}.

Dividing out n + 1, it’s enough to show:

(n + 1)^{2} = 2(Σ_{i=1}^{n}i) + (n + 1).

Now it’s useful to substitute in the closed form proved above for the sum, so it remains to check:

(n + 1)^{2} = 2(\frac{n(n + 1)}{2}) + (n + 1).

This reduces by easy algebra.

6.4 Angles in A Polygon

Now an example of induction applied to geometry. Still simple induction. Still no need to strengthen hypothesis. Still numbers, though applied to geometry.

Definition 6.3 A convex polygon is a polygon such that any straight line between two polygon vertices does not leave the polygon.

Lemma 6.4 If we draw a line between two vertices of a convex polygon, it divides the polygon into two convex polygons.

Proof. Sketch. It is clear the line divides the convex polygon into two polygons. Each piece is convex because any straight line between two vertices of a piece lies entirely on one side of the dividing line and since it did not leave the original polygon, it must not leave the piece it is in.

Theorem 6.5 The sum of angles in an n-sided convex polygon is (n − 2)180^{∘}.

Proof. By induction. The induction hypothesis P (n) is the theorem statement. The base case, for n = 3, is true from Euclid. For the inductive step, assume P (n). To prove P (n + 1), consider an (n + 1)-sided polygon. Consider three vertices A, B, C that appear consecutively on the boundary of the polygon. Draw a line AC. This produces a triangle ABC and a polygon with n vertices. (We’ve removed B.) By the preceding lemma, the smaller polygon is convex and thus, by the induction hypothesis, contains (n − 2)180^{∘}. We now add in the 180^{∘} contained in the angles of ABC to find that the total angle sum in the (n + 1)-vertex polygon is (n − 2)180^{∘} + 180^{∘} = ((n + 1) − 2)180^{∘}, which proves P (n + 1).

6.5 Die Hard

Another applied induction. Simple induction. Now something new needed to strengthen hypothesis in order to prove.

In Die Hard III (Die Hard with a Vengeance), Bruce Willis and Samuel Jackson had to disarm a bomb using the old boy scout trick of measuring four gallons of water using a three-gallon jug and a five-gallon jug. (You can figure out how this goes.) The jugs are initially empty. They have access to a reservoir of water. They need to pour out four gallons into a special container. The only operations they are permitted are those of pouring water using the three-gallon jug and the five-gallon jug. Whenever they pour water from a jug, they either have to pour out all the water in the jug, or pour until the receptacle into which the water is being poured becomes full. They can also fill either jug from the reservoir.

In Die Hard IV, they again have to pour out four gallons, this time using a 3 gallon and a 6 gallon jug. The movie is not likely to have a happy ending, because:

Theorem 6.6 Using a 3 gallon jug and a 6 gallon jug and the operations above, it is not possible to measure 4 gallons.

Want to prove this by induction. But there is no obvious inductive hypothesis here. Could try: Q(n): After n operations, it’s impossible for either jug to contain exactly 4 gallons. Then Q(0) is fine – both are empty at first. Inductive step: Assume Q(n), prove Q(n + 1). ??? But just knowing Q(n) isn’t enough to prove Q(n + 1): Maybe it’s possible for the 6 gallon jug to contain 1 gallon after n operations. (The inductive hypothesis doesn’t rule this out – only rules 4 out.) If this can happen, then one more operation could yield 4 gallons.

However, it is possible to prove this theorem by induction. The trick, though, is to strengthen the property to be proved so that it:

1. Implies the property that we really want, and

2. Can be proved by induction.

Very important method. Here, the strengthened property that works is: P (n): After n operations, the number of gallons in each jar is divisible by 3.

Proof. Prove P by induction on n, the number of operations. BASE CASE: P (0) is clearly true since the containers are initially empty. INDUCTIVE STEP: Assume P (n) is true. Consider (n + 1) operations. At the end of the first n operations, by the inductive hypothesis, all containers contain a number of gallons divisible by 3. There are several possibilities for the (n + 1)st operation: A jug could be filled from the reservoir, or emptied into the reservoir, or one jug could be poured into the other. In any of these cases, the resulting number of gallons in both containers continues to be divisible by 3. This completes the proof by induction.

Since 4 is not divisible by 3 it is not possible to measure 4 gallons.

6.6 Courtyard Tiling

Another nice example where you need to strengthen the inductive hypothesis in order to prove it.

Consider 2^{k} × 2^{k} square courtyard, k ≥ 1. Want to tile it using 3-square L-shaped tiles, leaving a particular one of the four center squares untiled (so you can install a birdbath). You don’t care what the tile pattern looks like otherwise.

Theorem 6.7 Any 2^{k} by 2^{k} courtyard with any central square filled in can be tiled with L shaped tiles (for k ≥ 1).

Proof. Try by induction on k. The obvious is to try Q(k) being the statement above. Base: k = 1 is fine, because the rest is just a single L. Inductive: Obvious thing to try is to split a 2^{k+1} × 2^{k+1} courtyard into four pieces, using the assumed tilings for the pieces and then try to paste them together. But the assumed tiling for the pieces leave four isolated squares in the middle. Not too helpful...

It would be better if the left-over squares were in the corners of the four pieces. That way, could put the empty corners together and use one additional L, leaving one center square. But that’s not what the inductive hypothesis gives. ???

We can’t just use the hypothesis that we can tile except for an arbitrary corner. This can be proved inductively, but in the end, it doesn’t give the result we want (missing center square, not corner).

So, try: P (k): Any 2^{k} × 2^{k} courtyard can be tiled leaving any corner square, or any center square, open.

P (1) clear. Inductive step: Given a 2^{k+1}×2^{k+1} courtyard, use inductive hypothesis to get four 2^{k}×2^{k} pieces with missing corners. Must consider two cases:

1. The bigger courtyard is to have a missing center square: Then put the corners together in the middle, as above, cover three of them with one L.

2. Missing corner:

Then put three of the corners together, in the middle, the other in the desired corner. Cover the ones in the middle with one L.

An alternative inductive hypothesis that also works: R(k): Can tile a 2^{k} × 2^{k} courtyard with any tile missing. Sketch why: Given missing square, divide in four, fill in the component with the missing piece using inductive hypothesis. For the other three, put the missing square in the middle corner. Use an additional L.

Things to watch when inventing an inductive hypothesis:

• It needs to be provable by induction.

E.g., “can tile any square courtyard with any number of squares missing, with L tiles”, can’t be proved by induction, since it isn’t even true. (Why not?) E.g., “can tile any courtyard with center square missing” can’t be proved by induction, since it’s too weak for the induction step to go through.

• It needs to imply the result that you really want to prove. E.g., “can tile with corner missing” can be proved by induction, but doesn’t imply what we want.

How to come up with the strengthened hypotheses? No magic formula – it’s an art.

6.7 Algorithm Verification

A final use of induction today – verifying iterative algorithms.

Algorithm for multiplying a × b, where a and b are arbitrary length positive integers represented in binary, and result is to be represented in binary. Assume that we have integer addition available, and can examine individual bits. b(k) denotes kth high order bit of b.

Algorithm: Maintain accumulator acc, initially set to a. Suppose that b has n bits. For k = 1 to n − 1 do: if b(k + 1) = 0 then acc := 2acc else acc := 2acc + a

At end, result is in acc.

Example: a = 5, b = 13 = 1101.

k acc high(b, k + 1) 0 5 1 1 15 3 = 11 2 30 6 = 110 3 65 13 = 1101

How do we know this is correct? Can prove, by induction. For every k, 0 ≤ k ≤ n − 1, P (k), where P (k): After k iterations, acc is equal to a × high(b, k + 1).

Writing high(b, m) for the integer defined by the m highest-order bits of b.)

BASE: k = 0. The highest-order bit is 1, so high(b, 1) = 1. Initially, acc contains a = a × 1, as needed. INDUCTIVE STEP: Assume for k. After k iterations, acc contains a × high(b, k + 1). Show after one more iteration, acc contains a × high(b, k + 2).

Depends on examining the k + 1st iterative step. At this step, examine k + 2nd bit of b. Rule says if 0, double, else double and add a. Check cases to see this is the right thing to do: If 0, new desired number is just double the old. If 1, new desired number is double plus a, ...

Why does this say that the final answer is correct?

Specialize the result to k = n − 1, P (n − 1): After n − 1 iterations, acc is equal to a × high(b, n) = b.

6.8 Fallacies

There are several common mistakes made in inductive proofs. The most common is forgetting to prove the base case:

False Theorem 1 All horses are the same color.

Proof. Let P (n) be the proposition that all horses in any set of n horses are the same color. Clearly, P (1) is true. Now assume that P (n) is true. Consider any n + 1 horses; number these as horses 1, 2, 3, …, n + 1. The first n of these horses are the same color, and the last n are also the same color. Since the set of the first n horses and the set of the last n horses overlap, all of the n + 1 horses are the same color. This shows that P (n + 1) is true and finishes the proof by induction.

This proof is wrong because the inductive step does not work for n = 1, i.e. the truth of P (1) does not imply the truth of P (2).

Another common mistake is using an inductive hypothesis that is just too weak. E.g., in the tiling example, the hypothesis that you can tile with any center square missing. Another example of this sort is the following variant on the triangle angle-sum example.

Theorem 6.8 All regular convex polygons with n sides have angles summing to (n−2)180^{∘}.

Proof. False proof. The proof is by induction. Let P (n) be the proposition that all regular convex polygons with n sides have angles summing to (n − 2)180^{∘}. BASE CASE: P (3) is true since an equilateral triangle has angles summing to (3 − 2)180^{∘}. INDUCTIVE STEP: Assume P (k) to be true, for all k ≤ n. Consider a regular convex polygons with n + 1 sides. Consider three vertices A, B, C that appear consecutively on the boundary of the polygon. Draw a line AC. This produces a triangle ABC and a polygon with n vertices (we’ve removed B). By lemma 2, the smaller polygon is convex and thus, by the induction hypothesis, contains (n − 2)180^{∘}. We now add in the 180^{∘} contained in the angles of ABC to find that the total angle sum in the (n + 1)-vertex polygon is (n − 2)180^{∘} + 180^{∘} = ((n + 1) − 2)180^{∘}, which proves P (n + 1).

This completes the inductive step and hence P (n) is true for all nonnegative integers n.

The above proof is wrong because the inductive hypothesis has been invoked wrongly. The smaller polygon created by line AC is not regular. But it is almost the same proof as the right one.