Massachusetts Institute of Technology Lecture 3 6.042J/18.062J: Mathematics for Computer Science September 12, 1996 Professors Nancy Lynch and Tom Leighton

Notes 3

Recall the Peano induction rule:

Let P (n) be a predicate, and suppose

• P (0)

• (∀n) (P (n) ⟹ P (n + 1)).

Then P (n) is true for all natural numbers n.

Plus simple variation where lower limit isn’t 0, but some other natural number.

Today we describe two more interesting variations on the induction rule. Also valid for reasoning about natural numbers.

Strong induction and well-ordering.

Strong induction is nice because it allows you to use not just P (n − 1) when trying to conclude the result for P (n), but also P of anything smaller than n − 1, i.e., P (0), …, P (n − 1).

Well-ordering is a restatement of induction that says that every nomempty set of natural numbers has some smallest element.

We will also sketch proofs that these three induction rules are “equivalent”. Thus, they are interchangeable in arguments, though in particular practical cases, one may work out much nicer than the others.

6 Picky Axiomatic Remarks

When I give the new induction axioms, they will mention an ordering < on the natural numbers. The astute, picky student will observe that the Peano axioms don’t mention anything about an ordering; they just mention 0 and successor. And we haven’t defined any ordering yet. Only defined operations + and ×.

If we want to treat these as alternatives to the Peano induction axiom, we will have to add some new notions to the Peano axioms. This new stuff will include a new symbol <, and some new simple axioms like:

• < is a strict total order.

(Formally, irreflexive, ¬n < n; antisymmetric, ¬(m < n ∧ n < m); transitive, m < n, n < p imply m < p; total: any m, n have either m < n or n < m.)

• Every natural number other than 0 is a successor of something, i.e., n ≠ 0 ⟹ ∃m(m′ = n). (Call this its predecessor. It must be unique.)

• 0 < n for all other n.

• n < n′.

• m < n if and only if m′ < n′.

• ¬(m < n ∧ n < m + 1).

• Etc.

We’ll come back to this later in the hour. For now, we’ll just continue assuming such basic properties.

7 Strong Induction

7.1 Definition

A variant of induction. To prove (∀n ∈ N)P (n)

• prove P (0)

• prove ((∀k < n)P (k)) implies P (n).

Actually, don’t even need the base case. That’s because the second case, when specialized to n = 0, says to prove ((∀k < 0)P (k)) implies P (0).

But the hypothesis is vacuously true, since there’s nothing < 0. So this means you’re really forced just to prove the conclusion, P (0), on its own.

But never mind if this is confusing – we’ll follow the book and check base cases – they’re generally easy anyway. Another reason for base cases: Often the first case or first few cases need a different argument for the inductivve step, show explicitly.

Again, can modify this to work from any k instead of 0.

7.2 Fibonacci Examples

Example 1 Recall Fibonacci numbers:

• f_{0} = 0

• f_{1} = 1

• f_{n+1} = f_{n} + f_{n−1}

Theorem 7.1 f_{n} ≥ (\frac{3}{2})^{n−2} for all n ≥ 2.

Proof. Define P (n) as predicate above.

Suffices to show for all n ≥ 2: ((∀k, 2 ≤ k < n)P (k)) implies P (n). (No base case.)

But convenient to check P (2) and P (3) explicitly in the inductive step. Or you can regard them as base cases. At any rate, they use a different argument.

For n ≥ 4: Assume ((∀k, 2 ≤ k < n)P (k)), show P (n).

By definition, f_{n} = f_{n−1} + f_{n−2}. Both of these are ≥ 2, so we can assume P (n − 1) and P (n − 2), by the strong inductive hypothesis. That is, f_{n−1} ≥ (\frac{3}{2})^{n−3} and f_{n−2} ≥ (\frac{3}{2})^{n−4}.

Now, f_{n} = f_{n−1} + f_{n−2} ≥ (\frac{3}{2})^{n−3} + (\frac{3}{2})^{n−4}. We want to show that this is ≥ (\frac{3}{2})^{n−2}.

Dividing all terms by (\frac{3}{2})^{n−4}, we see that we need to show: \frac{3}{2} + 1 ≥ (\frac{3}{2})^{2}. This is easily checked.

Book does a sharper calculation.

Note how we used the inductive hypothesis about two earlier cases.

Another example of a mistake – forgetting to prove enough base cases to make the induction step go through.

Theorem 7.2 (False theorem) Every Fibonacci number is even.

Proof. The proof is by strong induction. Let P (n) be the proposition that f_{n} is even. BASE: Prove explicitly that P (0) is true – this is because f_{0} = 0 is even. INDUCTIVE STEP: Assume P (k) to be true, for all k < n. Consider f_{n} = f_{n−1} + f_{n−2}. Since f_{n−1} and f_{n−2} are even, by the inductive hypothesis, it follows that f_{n} is even too. Hence P_{n} is true.

This completes the inductive step and hence P (n) is true for all nonnegative integers n.

The above proof is faulty because there were not enough base cases shown explicitly. The inductive argument doesn’t work to show P (1), since f_{−1} is not defined. Would have to show P (1) explicitly, but of course that doesn’t work, since f_{1} is odd.

7.3 Number Theory

Definition 7.3 For a > 0, we say a divides b if there exists an integer t such that a = bt. We write a | b.

• 3 | 6

• 3 |̸ 7

Definition 7.4 n ≥ 2 is prime if the only natural numbers dividing n are 1 and n.

Definition 7.5 n ≥ 2 is composite if n = ab for natural numbers a, b ≠ 1, n.

Every natural number is either prime or composite. (Why?)

Theorem 7.6 Every natural number ≥ 2 has a prime divisor.

Proof. By strong induction. BASE CASE: The theorem is true for n = 2.

INDUCTIVE STEP: Assuming the theorem true for all k, 2 ≤ k < n, consider the number n. We proceed by cases:

1. n is prime. Then n (a prime) divides n.

2. n is composite. Then n = ab where 2 ≤ a < n. It follows from the inductive hypothesis that a has a prime divisor, ie that a = pt for a prime t. Then n = ptb, meaning p divides n.

Since we proved the inductive step for both possible cases, we are done.

Note: we use a base case of 2.

7.4 Structural Induction on a Recursive Definition

Induction is often used to prove properties for all elements of a set defined recursively, like well-formed arithmetic formulas or binary trees. This is usually formulated so it looks like a different kind of inductive method: Two steps:

• Prove for “base cases” of definition

• Prove for the result of any combination rule, assuming it’s true for parts.

But this isn’t really different. It’s an instance of strong induction on the number of formation steps. Just say that the base cases correspond to 0 applications of combination operations.

Give two examples here.

Theorem 7.7 The number of edges in any binary tree is exactly one fewer than the number of nodes.

Proof. The proof is by strong induction on n, the number of applications of the “combination rules”. Let P (n) be the proposition that a binary tree formed by n applications of the combination rules has 1 fewer edge than nodes.

BASE CASE: P (0) is true since a binary tree that is a node with no children has 0 edges.

INDUCTIVE STEP: Assume P (k) to be true, for all k < n. Consider a binary tree formed by n applications of the combination rules. There are three cases.

1. The tree is a node with a left child that is a binary tree.

Since the left child is a binary tree formed by n − 1 applications of the combination rules, by the inductive hypothesis, it has 1 fewer edge than nodes. Hence the whole tree has 1 fewer edge than nodes, too. (Adding one edge and one node.)

2. The tree is a node with a right child that is a binary tree.

This case is identical to the previous case.

3. The tree is a node with left and right binary-tree children.

Then each of the two subtrees is formed by fewer than n applications of the combination rules. By inductive hypothesis (using strong induction here, since it needn’t be n − 1), each subtree has exactly 1 fewer edge than nodes. The whole tree adds 1 node and 2 edges, so also has exactly 1 fewer edge than nodes.

This completes the inductive step and hence P (n) is true for all nonnegative integers n. This implies the result.

Theorem 7.8 Well formed formulas always have the same number of left and right parentheses.

Proof. The proof is by strong induction on n, the number of applications of the “combination rules”. Let P (n) be the proposition that a well formed formula formed by n applications of the combination rules has the same number of left and right parentheses.

BASE CASE: P (0): If the formula is just a number then the above proposition is obviously true since there are 0 left and right parentheses. Hence, P (0) is true.

INDUCTIVE STEP: Assume P (k) to be true, for all k < n. Consider a well formed formula formed by n applications of the combination rules. There are four cases.

1. The formula is of the form (f + g), where f and g are well formed. Since f and g are well-formed and are formed by fewer than n applications of the combination rules, by the strong inductive hypothesis, they have an equal number of left and right parentheses and hence so does (f + g).

2. The formula is of the form (f − g). This case is similar to the first.

3. The formula is of the form (f ∗ g). This case is similar to the first.

4. The formula is of the form (f/g). This case is similar to the first.

This completes the inductive step and hence P (n) is true for all nonnegative integers n.

7.5 Tiling

Problem: What 2-dimensional planes can be tiled using only 1 × 3 and 2 × 2 tiles? Rotations allowed.

Easy case: What about 1 × n and n × 1? If and only if n is a multiple of 3. Easy, LTTR to prove carefully.

Next case: 2 × n (and n × 2) Claim any, if n ≥ 2. Show by strong induction:

BASE: 2 × 2, one tile. BASE: 2 × 3, two tiles.

INDUCTIVE: n ≥ 4. If n even, then use one 2 × 2, what’s left is 2 × (n − 2), where n − 2 ≥ 2. So by strong inductive hypothesis, can tile the rest.

If n odd, then n ≥ 5. Then use two 1 × 3’s to reduce to 2 × (n − 3), where n − 3 ≥ 2. Again, strong inductive hypothesis gives the rest. Yet another case: 3 × n (and ...) Then all, obviously.

Finally: What about m × n, m, n ≥ 4? Claim all.

Proof by strong induction on m (where P (m) says “for all n ≥ 4, can tile m × n”.

Cases: m even, m ≥ 4: Then first two rows are 2 × n, n ≥ 4, can be tiled by special case lemma. Remove, get (m − 2) × n. If m − 2 = 2 then can do rest by special case lemma. Else m − 2 ≥ 4, can do rest by inductive hypothesis.

m odd, m ≥ 4: So m ≥ 5. Then first three rows can be done, by special case lemma. Remove, get (m − 3) × n. If m − 3 = 2 then can do rest by special case lemma. Else m − 3 ≥ 4, can do rest by inductive hypothesis.

7.6 Ordinary vs. Strong Induction

What is the relationship between the ordinary and strong induction axioms? It should be clear, intuitively, that the strong induction axiom is at least as powerful as the ordinary one – any proof you can do with weak induction still works based on strong induction. You only need to invoke the inductive hypothesis for n − 1, whereas you’re allowed to use all cases smaller than n.

(Remark: This assumes existence of a predecessor, which follows from our basic axioms.)

Though it’s not obvious, the strong induction axiom is really no stronger than the weak axiom! That is, any proof you can do based on strong induction can be reformulated so it just works using weak induction. You just have to use weak induction in a tricky way: You must strengthen the predicate you prove using weak induction so that it includes information about smaller arguments.

You should be used to strengthening inductive hypotheses...

Suppose P is a predicate such that (∀nP (n)) can be proved using strong induction.

Means: P (0) (∀k < n)P (k) ⟹ P (n)

Show that we can prove using ordinary induction too. To do so we need to strengthen P so the ordinary induction works:

Define Q(n) = (∀k ≤ n)P (k)

Claim: can prove Q using ordinary induction

Base: Q(0)– Save as P (0), already know this Inductive step:

Show that Q(n)impliesQ(n + 1) Show that (∀k ≤ n)P (k) implies (∀k ≤ n + 1)P (k) We know this, since (∀k ≤ n)P (k) ⟹ P (n + 1)

Claim: Q(N) ⟹ P (n)foralln So, by strengthening to Q, we have proved P using ordinary induction.

Formally: If you start with a basic set of axioms – not just Peano, but including ordering axioms and existence of predecessor as above – then adding either induction axiom allows you to prove the other.

Theorem 7.9 Weak induction axiom implies strong induction axiom.

Proof. Suppose we have the weak induction axiom. This says, for any predicate P, that P (0) and (∀n)(P (n) ⟹ P (n + 1)) together imply (∀n)P (n).

What we want to prove is the strong induction axiom, which says:

For any predicate Q, Q(0) and (∀n)(((∀k < n)Q(k)) ⟹ Q(n)) together imply (∀n)Q(n).

So fix some Q that we’d like to prove using strong induction. And assume ** Q(0) and (∀n)(((∀k < n)Q(k)) ⟹ Q(n)). ** We will see how to prove (∀n)Q(n) using ordinary induction.

The key is to come up with just the right strengthened predicate P to make an ordinary induction go through. Here’s the clever trick:

Define P (n) as the predicate (∀k ≤ n)Q(k). Thus, P (n) asserts not only Q(n) for the same argument n, but also for all smaller arguments k < n. Need to strengthen it in this way to make the induction go through using the weaker principle.

Now check that this definition works. Straightforward, but can be a little hard to keep straight.

In order to apply weak induction, we must show two things: P (0) and (∀n)(P (n) ⟹ P (n + 1)). We’ll verify these two, then apply weak induction and see what we get.

P (0): This says (∀k ≤ 0)Q(k). That’s the same as just Q(0). But we’ve assumed this to be true, *** above.

P (n) ⟹ P (n + 1). (Fix n.) Assume P (n). That says (∀k ≤ n)Q(k). By what we have assumed about Q *** above, we know that ((∀k < n + 1)Q(k)) ⟹ Q(n + 1). Therefore we know Q(n + 1). So, we have (∀k ≤ n + 1)Q(k). But this is just P (n + 1), as needed.

Now we can apply ordinary induction to P, concluding that P (n) holds for all n. But P (n) ⟹ Q(n), so also Q(n) holds for all n.

8 Well Ordering Principle

8.1 Definition

The well ordering principle says: Every nonempty set of natural numbers contains a least element. This should be “obvious”, as a property of natural numbers as commonly understood.

This may not look exactly like an induction principle, but it’s similar. Formally, it’s equivalent (in the presence of basic axioms) to induction (weak or strong). Come back to this later.

Well ordering not true for integers, positive reals, positive rationals.

8.2 Number Theory

We give some number theory examples of the use of well ordering.

Theorem 8.1 Every natural number ≥ 2 can be factored as a product of primes.

Proof. By well ordering. Let S be the set of all natural numbers ≥ 2 that cannot be factored as a product of primes. We assume S ≠ ∅ and derive a contradiction.

Since S is not empty, well ordering implies that it has a least element n.

This element n cannot be a prime, because then it would be a (trivial) product of primes. So n is composite – write n = ab where 1 < a, b < n.

Because a, b < n, and n is the smallest element of S, it follows that a, b ∉ S. This implies that a can be written as a product of primes, say p_{1}p_{2} ⋯ p_{k}, and b as a product of primes, say q_{1} ⋯ q_{l}.

Therefore, n can be written as p_{1} ⋯ p_{k}q_{1} ⋯ q_{l}, a product of primes, contradicting the claim that n ∈ S.

Our assumption that S ≠ ∅ must therefore be false.

It is also possible to prove this result using strong induction. The argument is very similar, in fact. (Check this.)

This similarity should give you the idea that the well ordering principle is very similar to strong induction. In fact, it’s equivalent (see later).

A fallacious argument: Try to show that every number can be factored uniquely into primes.

Apply the same (well ordering) proof as above, modifying the definition of S: Let S be the set of all natural numbers ≥ 2 that cannot be factored in exactly one way as a product of primes.

The fallacy is that even if n = ab and a and b have unique factorizations, it is still possible that n = cd for different c and d, producing a different factorization of n.

The argument is false, but the claim is true and is known as the fundamental theorem of arithmetic.

Theorem 8.2 (The division algorithm.) Let a, b be nonzero naturals. Then ∃!q, 0 ≤ r < b, such that a = qb + r.

Proof. To prove this theorem, we need to prove both that some r exists and that this r is unique (this is typical of unique-existence proofs). Fix a, b.

Existence: We use well ordering.

Let S = {r ∈ ℕ | (∃q ∈ ℕ)a = qb + r}.

Example:

If a = 13 and b = 3, then S = {1, 4, 7, 10, 13}.

S is nonempty because a = 0b + a implies a ∈ S.

It follows by well ordering that S contains a least element r_{0} such that a = q_{0}b + r_{0}. We claim that this (q_{0}, r_{0}) pair has all the properties we need.

The only interesting thing to check is that r_{0} < b.

We argue by contradiction: suppose r_{0} ≥ b. Then a = (q_{0} + 1)b + (r_{0} − b) and r_{0} − b ≥ 0, which implies that r_{0} − b ∈ S. But this contradicts the assumption that r_{0} is the least element of S. Thus r_{0} < b.

Uniqueness: Suppose a = q_{1}b + r_{1} = q_{2}b + r_{2}.

Without loss of generality, suppose r_{1} > r_{2}.

(What we mean is, if r_{1} < r_{2}, then we can swap (q_{1}, r_{1}) with (q_{2}, r_{2}) to make the assumption true. Of course, we cannot have r_{1} = r_{2}.) Then subtract: (q_{1} − q_{2})b = (r_{1} − r_{2}). This means that b | r_{1} − r_{2}. By assumption, 0 ≤ r_{1}, r_{2} < b. So 0 ≤ r_{1} − r_{2} < b. Therefore, r_{1} − r_{2} = 0. So q_{1} − q_{2} = 0.

Definition 8.3 In the theorem above, q is the quotient and r is the remainder.

Another example: We left a slight hole in our proof that \sqrt{2} is irrational – we assumed that we could write a fraction in lowest terms, that it, in such a way that the numerator and denominator have no common divisors. Here is how to actually prove that fact:

Theorem 8.4 Every rational number can be written in lowest terms.

Proof. Fix rational q. Let S = {b ∈ ℕ | (∃a)q = a/b}. (The set of possible denominators.) S is nonempty, by the definition of q being rational. Therefore S has a least element b and a corresponding a. Then a/b is the lowest-terms representation of q.

8.3 Nim

The game of Nim is defined as follows. A bunch of sticks are placed on the ground. Two players take turns removing one, two, or three sticks. The player to remove the last stick loses.

Theorem 8.5 First player loses if the number of sticks n is 4k + 1 for some k, and wins (if he plays optimally) otherwise.

Proof. By well ordering. Let S be the set of numbers n for which the theorem is violated, that is, either n = 4k + 1 and the first player wins, or else n = 4k, 4k + 2, 4k + 3 and the first player loses. Assume S is nonempty. Let n be the smallest element of S.

There are four cases:

• n = 4k + 1 and the first player wins.

Can’t have n = 1, because the first player has no choice but to remove 1 stick and lose. So must have n ≥ 5.

Consider what the first player might do to win. Can only choose to remove 1, 2 or 3 sticks.

If removes one stick, the remainder is n − 1 = 4k. By choice of n as smallest element of S, the player who plays at this point should win. It follows that the player who played first will lose, contradiction.

Similarly, if the first player removes two sticks, the remainder is 4(k − 1) + 3, and he loses by the same reasoning. Another contradiction.

Similarly again for removing 3 sticks.

So however the first player moves, a contradiction is reached. This means that the smallest violating element n can’t be of the form 4k + 1.

• n = 4k and the second player wins.

Then if the first player removes three sticks, the second player sees 4(k − 1) + 1 sticks.

This means he loses, because n is the smallest element of S. Contradiction.

• n = 4k + 2 and the second player wins.

Similar to the previous case.

• n = 4k + 3 and the second player wins.

Again similar.

From the division algorithm, we know these are all the possible cases.

Again, easy to reformulate using strong induction.

8.4 Well Ordering vs. Induction

In the presence of basic axioms about numbers, well ordering turns out to be equivalent to induction (weak or strong, since they are equivalent to each other).

Theorem 8.6 Induction implies well ordering.

Proof. We must show: Every nonempty set contains a smallest element. We can assume either kind of induction axiom, since they are equivalent. So we’ll use strong induction, since that gives us more power.

Define predicate P (n): Every set containing n contains a smallest element.

If we prove that P (n) is true for all n, then we have well ordering since a nonempty set has to contain some element n.

Prove this by strong induction: Basis: P (0): If a set contains 0, then 0 is its smallest element. Follows from basic ordering properties.

Inductive step: Assume (∀k < n)P (k), and prove P (n). Consider a set containing n. If it contains no smaller element, then n is the smallest element and we are done. On the other hand, if it does contain a smaller element k < n, then by the inductive hypothesis, it contains a smallest element and again we are done.

Theorem 8.7 Well ordering implies induction.

Proof. We will show it implies weak induction.

We have to show that for any predicate P, P (0) and (∀n)(P (n) ⟹ P (n + 1)) together imply (∀n)P (n).

So fix P and assume P (0) and (∀n)(P (n) ⟹ P (n + 1))

We have well ordering, which says any nonempty S has a smallest element. We need to define the right S so we can apply well ordering: Define S = {n | ¬P (n)}. The set of elements not satisfying P.

Suppose that S ≠ ∅ and derive a contradiction.

If S ≠ ∅ then, by well ordering, it contains a smallest element n. Clearly n ≠ 0 since P (0) is true. Therefore n − 1 is a natural number (uses basic order axioms) and it is not in S (since n is the smallest element). So P (n − 1) is true. It follows from weak induction that P (n) is true. which contradicts the claim that n ∈ S.

We have derived a contradiction, meaning that our initial assumption, that S is nonempty, is false. So S is empty, which implies that P (n) is true for all n. As needed.