Massachusetts Institute of Technology Lecture 1 6.042J/18.062J: Mathematics for Computer Science September 5, 1996 Professors Nancy Lynch and Tom Leighton

Notes 1

1 Introduction

1.1 Class Content

• Introduces the most important discrete (define) mathematical structures for modelling computer systems.

Numbers, sets, graphs, state machines, probability distributions, ...

• Introduces the most useful mathematical techniques for reasoning about these math structures.

Propositional and predicate logic, induction, recurrences, generating functions, ...

• Teaches notion of mathematical proof as a discipline for reasoning carefully about such structures.

Does this by doing proofs carefully for virtually everything covered in the course. In class, for homework. Also lots of examples of pitfalls.

• Teaches how one can abstract from complex application scenarios to represent/analyze real problems using all this math.

We will begin by introducing basic notions of proofs, just using simple, familiar structures like sets, natural numbers. Then we will introduce more structures. Then methods. Then probability – postponed to end because it’s harder, do as a long unit.

Proofs throughout.

(Go over course outline briefly.)

2 Basic Proof Notions

2.1 Propositions

A proposition is a statement that is either true or false but not both.

Knowledge consists of propositions, such as • 2+3=5

• For every a, b and c, if a = b and b = c then a = c

• The sum of the interior angles of every triangle is 180^{∘}

• Every even integer greater than 2 is the sum of two prime numbers (Goldbach’s conjecture).

• For all n ≥ 1, 1 + 2 + ⋯ + n = n(n + 1)/2

• For all n ≥ 0, n^{2} + n + 41 is a prime number.

• With perfect play, chess is a win for white.

• The Pentium chip divides properly.

• Every map can be colored with four colors, so that no two adjacent regions have the same color (Four Color Theorem).

• With a fair coin, the probability of getting 50 heads in 100 tosses is 1/2.

Each of these propositions is either true or false. We (and Intel) would like to know which. [[[Too old joke?]]] Notes:

• Not all sentences are propositions:

– +2 ∗ ∗ = 3 [[[What’s this?]]]

– What time is it?

– This proposition is false.

• Propositions involve previously defined terms:

– prime, map, triangle, chess, fair coin, probability – +, =, 1, ≥

2.2 Proofs – What (Informal) and Why

Layman’s definition of proof: “A method of ascertaining the truth.” Many ways to decide what’s true:

• evidence (eg, general relativity, big bang),

• reputable authority (eg, Professor),

• inner conviction (eg, religion), But these methods can go wrong:

• Evidence: n^{2} + n + 41. Program works fine on all test cases, but…,

• Authority: Intel says Pentium is fine

• Conviction: There aren’t any bugs in my programs…

So mathematics focuses on a particularly convincing kind of argument: a proof. What is a proof? In math, it is a verification of a proposition by a chain of logical deductions from a base set of axioms. An axiom is a proposition that is assumed to be true.

• Begin with true propositions.

• Combine using specific inference rules.

• Derive more true propositions.

Should know and admit all unproven assumptions. Example:

Theorem 2.1 \frac{d}{dx} ln(3x + 1) = \frac{3}{3x+1}

Proof.

\frac{d}{dx}1 = 0
\frac{d}{dx}3x = 3
\frac{d}{dx}(u + v) = \frac{d}{dx}u + \frac{d}{dx}v

Combine, with u = 3x, v = 1:

\frac{d}{dx}(3x + 1) = 3
\frac{d}{dx} ln u = \frac{1}{u} \frac{du}{dx}

Combine, with u = 3x + 1.

This was a direct proof: start with knowns, plug together till you get desired result. Why do proofs?

• Convince yourself that your reasoning is correct – that you haven’t made mistakes.

• Convince others.

• A good proof has considerable structure, like a good program. Much of it can be reused for other related results (like a well-written program).

• A proof pins down why something is true, so you can know whether it will stay true when things change.

• A failure to prove something often reveals the reason it is false.

Program verification: proof that a program does the right thing.

• Pentium

• Airbus A300

• Therac 25

[[[The examples above are from David Karger.]] Other examples:

• Kaashoek’s Orca system, for which Alan Fekete and I helped to verify the main algorithm (and found a significant logical error in the process of doing the proof).

• Current chip designs – companies like Intel, HP are now investing in verification people and computer tools.

• Communication protocols, AT&T.

• Practical verification efforts are also beginning for real-time process control systems, e.g., those systems that control nuclear power plants and airplanes. Still preliminary.

Program verification is a nice complement to testing – testing can often detect bugs quickly. Verification is harder, but it does help to find deeper logical mistakes. And when you’re done, you know there are no more errors (which testing alone can never guarantee).

2.3 Basic Concepts/Notation

2.3.1 Sets

A set is a collection of objects:

• x ∈ S means “x is in the set S”

• ∅ is the empty set (contains no elements).

• A ∩ B means “the intersection of A and B” (the set of elements in both sets).

• A ∪ B means “the union of A and B” (elements in either set).

• Set difference. A − B = {x|x ∈ A, butx ∉ B}

• ⊆, subset.

• =, set equivalence.

We won’t be axiomatic about sets (“naive set theory”).

2.3.2 Boolean Operators

Boolean symbols are useful for brevity and precision, just like arithmetic symbols. If

• p denotes “I like chocolate”

• q denotes “I like ice cream”

then

• p ∧ q means “p and q” (I like chocolate and I like ice cream). True iff p and q are both true.

• p ∨ q means “p or q.” True iff one or both of p and q is true. Not an exclusive or. (How could I formulate exclusive or?)

• p ⟹ q means “p implies q” or “if p then q” (If I like chocolate, then I like ice cream). True iff p being true makes q true.

• note: p ⟹ q is certainly true when p is false!

• p ⟺ q

[[[Go over truth tables for all these operators. Make sure they know that p ⟹ q is true when p is false. Might be worth doing an example or two, calculating the truth values for some formula, showing a formula is a tautology, showing that two formulas are equivalent. Define contrapositive and assert equivalence with original proposition. (p ⟹ q) ⟺ (¬q ⟹ ¬p) Converse, inverse?]]] 2.3.3 Predicates

Predicates are similar to propositions but they depend on variables. P (x, y, z) denotes a predicate on variables x, y, z. For example, define:

• R(x): x is an odd number

• Q(x): x > 2

• P (x): x is a prime number

These predicates are not propositions, because they can be true or false depending on x. They are like functions from variables to truth values.

• P (2) is true

• R(2) is false

• P (4) ∨ R(3) is true

• Q(3) ∧ R(4) is false

• Q(1) ⟹ R(2) is true.

2.4 Quantifiers

Many propositions assert that something is true for all or some elements of a given domain (e.g., all numbers). Let’s agree in advance on a universe of discourse.

• “∀” means “for all.” ∀xP (x) is a statement that is true if P (x) is true for every x in the universe of discourse.

• “∃” means “there exists.” ∃xP (x) is true if P (x) is true for at least one x in the universe of discourse.

• “∃!” means “there exists exactly one.”

• “∀x ∈ S” means “for every x in the set S.” (∀x ∈ S)P (x) is equivalent to (∀x)(x ∈ S ⟹ P (x)).

Quantifies can turn predicates into propositions, e.g.

∀x, P (x) ∧ Q(x) ⟹ R(x)

This expression is a (true) proposition, because the variable in the predicates is bound. ∀x∃y is different from ∃y∀x:

• S(x, y) : student x falls asleep in class y.

• ∀y∃xP (x, y) : “In every class, some student will fall asleep”

• ∃x∀yP (x, y) : “There exists a student who will fall asleep in every class.”

• what is my universe of discourse above?

2.5 Axioms

Axioms are propositions that are assumed to be true.There doesn’t exist a proof that the axiom is true – you just start with a reasonable assumption. (“axiom” comes from Greek “to think worthy,” not “to be true”). It is not wrong to make assumptions (in fact, it is unavoidable), but to be rigorous you have to state your assumptions explicitly.

Some typical examples of axioms from various parts of mathematics:

• If a = b and b = c then a = c.

• Euclidean geometry has an axiom: Given a line l and a point p not on l, there is exactly one line through p parallel to l. (Sometimes an axiom from one domain contradicts an aiom from another.)

• Spherical geometry has an axiom contradicting Euclid’s: Given a line l and a point p not on l, there is no line through p parallel to l.

• So does hyperbolic geometry: Given a line l and a point p not on l, there are infinitely many lines through p parallel to l.

None of these axioms is “better” than any other: in fact, all yield equally consistent logical proof systems.

• Modern mathematicians still argue about which axioms are “right,” e.g., the Axiom of Choice (whose affirmation and negation are equally consistent).

Of course, different axioms lead to very different theorems. But anyone who agrees with your axioms and inference rules has to accept your theorems.

Once you have axioms and some prepositions that are believed to be true all you need is rules for combining true proposition to form new true propositions. Such rues are called ĭnference rules. The most famous rule is modus ponens:

• Let p and q be propositions.

• If p is true and p ⟹ q is true then q is true.

8 Lecture 1: Notes 1

2.6 Inference Rules for Propositions

When does one proposition follow from another? Let p and q be propositions. Then if p is true, and p ⟹ q (read p implies q) is true, we can conclude that q is true. This is a rule called modus ponens. Modus Ponens: \begin{matrix}p \\ p ⟹ q \\ q \end{matrix}

Syllogism: \begin{matrix}p ⟹ q \\ q ⟹ r \\ p ⟹ r \end{matrix}

Modus Tolens: \begin{matrix}p ⟹ q \\ ¬q \\ ¬p \end{matrix}

Proof by cases: \begin{matrix}a ∨ b \\ a ⟹ q \\ b ⟹ q \\ q \end{matrix}

2.7 Inference Rules for Predicates

Negation:

• (∀x)S(x) is equivalent to ¬(∃x)¬S(x)

– (∀x)S(x) : For all primes x > 2, x is odd.

– ¬(∃x)S(x) : There is no prime x > 2 such that x is not odd.

• (∃x)S(x) is equivalent to ¬(∀x)¬S(x). Universal generalization is a way to prove ∀xP (x):

• “Let x ≥ 2. Then x is prime or composite.”

• Conclude: ∀x ≥ 2, x is prime or composite (explain ∀x ≥ 2). We often leave out universal quantifiers if they are implicit. Like axioms, inference rules are an agreement to believe certain inferences. For example, the intuitionists (e.g. Brouwer) reject the claim that ¬∀x¬P (x) implies ∃xP (x). They claim that to prove something exists, you actually have to construct it.

Lecture 1: Notes 1 9

2.8 More Formally: Logical Proof Systems

A logical proof system consists of:

• A language for stating propositions

• Axioms: Propositions accepted as true

• Rules of inference: Acceptable ways of combining propositions

• Theorems: Propositions that can be derived by applying the inference rules to the

axioms. Note: we must always have some axioms: there are no “self evident” propositions. A logical proof system is

• consistent if no statement and its complement can both be derived.

• complete if it can be used to prove or disprove every proposition. Russell and Whitehead, Principia Mathematica, tried to completely axiomatize mathematics but failed. Gödel’s Theorems:

• no axiomatization of arithmetic is complete and consistent.

• “arithmetic is consistent” is a proposition that cannot be proved in arithmetic.

• Gensen: “arithmetic is consistent” can be proved by adding some reasonable-sounding

axioms, but these axioms can’t be proved consistent. There are true facts in math that cannot be proven! There are lots of ways to prove things, we now give some examples.

2.9 Direct Proof

Direct proofs are the most straightforward: we mix together previous axioms and theorems until the desired claim is inferred. For natural numbers a and b, where a ≠ 0, we say a divides b if there exists a natural number t such that at = b. We write a | b.

• 3 | 6

• 3 |̸ 7

[[[Tom: I changed the above to “natural number”, though the usual definitions are made for integers. This is because we haven’t mentioned integers yet. Maybe we should be listing some useful arithmetic data types earlier.]]]

Theorem 2.2 If a | b and a | c, then a | b + c.

Proof. By definition of divisibility, (∃t)b = at and (∃s)c = as. Therefore, b + c = at + as = a(t + s). Therefore, by definition, a | b + c.

Note: distributivity was assumed without mention.

2.10 Proof by Contradiction

\frac{p ⟹ F ALSE}{¬p}

Proofs can go “backwards”:

• Begin by assuming the desired theorem is false.

• Using inference rules, conclude something known false.

• Conclude that the assumption is wrong.

Theorem 2.3 \sqrt{2} is an irrational number.

Proof. (Indirect). Assume \sqrt{2} is rational.

Conclude \sqrt{2} can be written as a fraction a/b in lowest terms. Then 2b^{2} = a^{2}. In other words, a^{2} is even (since 2b^{2} is even). Therefore, a is even (the square of an odd number is odd). Therefore, a^{2} is divisible by 4. But b is odd (since a/b is in lowest terms and a is even). Thus 2b^{2} is not divisible by 4. Thus 4 divides a^{2}, but not 2b^{2} = a^{2}, contradiction.

Since our assumption leads to a false conclusion, it must be false. Thus, \sqrt{2} is not rational.

Note: this process is based on the assumption that math is consistent, since otherwise it would be possible to prove something false.

We freely used lots of basic facts about divisibility, lowest terms for fractions, definitions of square, all without reference or proof.

Pythagoras and his followers knew this proof, but suppressed it because they axiomatically denied the existence of irrational numbers (so their system was not consistent, because they also had an axiom that every line had a numeric length).

2.11 A Nonconstructive Existence Proof

Theorem 2.4 For any number n, there is a prime number greater than n.

Proof. Fix n, and consider n! + 1 = (1 · 2 · 3 ⋯ (n − 1) · n) + 1. If n! + 1 is prime, we are done. Otherwise, n! + 1 has a prime factor x (relies on the prime number theorem). By definition, n! + 1 is divisible by x (without leaving a remainder). But if n! + 1 is divided by a number less than n, the remainder is 1. Therefore, x > n.

(Note: n! + 1 need not be prime. 5! + 1 = 11^{2}.)

We don’t know what x is, only that it exists.

Intuitionists would deny this is a proof.

2.12 Proof by Cases

Proof by cases:

• show p ⟹ (a ∨ b)

• show a ⟹ q and b ⟹ q

• deduce p ⟹ q

Theorem 2.5 (∀xP (x)) ∨ A implies ∀x(P (x) ∨ A)

Proof. To prove the theorem, we need to show that if the hypothesis is true, then the consequent is true. So suppose the hypothesis is true. There are two cases: either A is true or A is false.

Case 1: A if true. If A is true, then P (x) ∨ A is true regardless of the truth of P (x). Thus, ∀x(P (x) ∨ A) is true.

Case 2: A is false. If A is false, then the only way for the hypothesis to be true is for the other disjunct, namely (∀xP (x)), to be true. But if P (x) is always true then certainly P (x) ∨ A is always true, in other words ∀x(P (x) ∨ A).

[[[Tom: I think the timing is iffy. This may run out of time around here. Is this OK? Should the rest be finished in recitation? There’s nothing hard remaining.]]]

2.13 Other Proof Methods

• Indirect proof: from ¬q ⟹ ¬p, deduce p ⟹ q (similar to proof by contradiction).

• Vacuous proof: if p is false, then p ⟹ q is true.

– if x is an even prime greater than 2, then x is a perfect square.

2.14 Fallacies

There are several ways to generate an incorrect proof.

\sqrt{−1} = \sqrt{−1}
\sqrt{\frac{1}{−1}} = \sqrt{\frac{−1}{1}}
\begin{matrix}√ & \overline{1} \\ √ & \overline{−1} \end{matrix} = \begin{matrix}√ & \overline{−1} \\ √ & \overline{1} \end{matrix}
\sqrt{1} · \sqrt{1} = \sqrt{−1} · \sqrt{−1}
1 = −1

Where’s the bug? There are several common fallacies:

Affirming the conclusion: • If you are tired, you will fall asleep • You fall asleep

• therefore, you are tired The fact that the conclusion is true does not make the hypothesis true. Maybe you are just bored.

Denying the hypothesis: • If you work hard, you will pass the course • You don’t work hard

• therefore, you will fail the course.

Even if the hypothesis is false, the conclusion can be true. Maybe you are smart enough to pass without working.

Circular reasoning: proposition: If n^{2} is not divisible by 3, then n is not divisible by 3.

• If n^{2} is not divisible by 3, then n^{2} is not equal to 3k for any integer k.

• therefore, n does not equal 3l for any integer l.

• therefore, n is not divisible by 3.

If you assume (or prove) something FALSE, you can prove anything!

3 How To Prove Things

Now we discuss how proofs are actually done in practice. Sketches that it’s supposed to be possible to complete.

We discuss some proofs that have arisen in actual math practice. Also algorithm proofs.

Then say what we actually expect in this course.

Then start doing through a catalog of practical proof methods. Some simple ones today, next two days concentrate on induction.

3.1 Some Actual Proofs...

Rigorous logical proof systems are nice, but no one really has the time or energy to be so careful. We prove things in a less formal way. The objective is to convince everyone that a truly rigorous proof could be built if someone took the time to do it. As a result, even great mathematicians can goof proofs.

• Andrew Wiles and Fermat’s Last Theorem

• Kempe’s invalid “proof” of the Four Color Theorem stood for 10 years (1879–1890)

• Gauss’s 1799 Ph.D. thesis is usually referred to as being the first rigorous proof of the Fundamental Theorem of Algebra (every polynomial has a zero over the complex numbers). But it contains quotes like

”If a branch of an algebraic curve enters a bounded region, it must necessarily leave it again. ... Nobody, to my knowledge, has ever doubted [this fact]. But if anybody desires it, then on another occasion I intend to give a demonstration which will leave no doubt.”

Fields Medalist Steve Smale writes about this, calling it an ”immense gap” in the proof that was not filled in until 1920, more than a hundred years later.

• In 1900 Poincare carelessly claimed a certain very simple topological characterization of the 3-dimensional sphere. Later realizing it was not so obvious, he demoted the claim to the status of a ”conjecture” in 1904. The Poincare Conjecture is now one of the biggest open questions in mathematics (two Fields Medals have been given out for partial progress on it).

• In the 1940’s Paul Erdos conjectured a certain combinatorial fact about arithmetic progressions (every set of natural numbers of positive density contains arbitrarily long arithmetic progressions), and offered $1000 for a proof or disproof. Endre Szemeredi proved the conjecture in 1972, but the proof was so long and badly written that Erdos was not sure whether to believe it. Finally, Andras Hajnal stepped in and wrote a much clearer version of the same proof, which was eventually published under Szemeredi’s name. At some point during the write-up, Hajnal informed Erdos that he was now confident enough of the proof that he was willing to buy it from Szemeredi for $500.

• There’s also the story of Newton waiting 20 years to publish the Principia because he didn’t see how to prove one could assume all the mass of a sphere to be concentrated at its center when calculating gravitational attraction. The proof ultimately required the invention of calculus ...

In programming, the current state of the art is that people do not actually prove correctness of running programs. There has been a lot of research directed toward making this possible, but it hasn’t yet gotten far enough to be practical. The problem is that the concepts that have to be reasoned about seem too low-level, complicated in their details.

In contrast, people do know how to model programs/systems abstractly, and prove correctness of the abstract versions. E.g., the algorithm used in a communication protocol can be described formally, and its correctness proved. But there is still a gap between what is verified and the actual code that runs on real systems – programmers implement algorithms in a rather ad hoc manner, and errors can creep in there. There isn’t much available now to help programmers bridge this gap.

The high-level algorithm proofs are usually done as sketches. Some are very complicated, and some errors have been found in published proofs. There is also considerable research in actually doing complete algorithm proofs axiomatically, and having a computer tool (an “automatic theorem-prover”) help in checking or even filling in the low-level steps. This sort of work is becoming practical.

Whether done by hand or with computer assistance, algorithm proofs are of great help in getting your programs right (and understanding why). I think this will be done much more in the future. Will become an important part of quality assurance for programs. It takes a lot of work to do it. And you have to understand formal logic proofs to do it.

3.2 What we Expect in this Course

Basic objective: like a scientific experiment, it has to be possible for someone else to “replicate” (ie, understand) your proof.

In the real world, anything that can be cited can be used without proof. But in homework, where everything we ask you to prove is already known, this defeats the purpose (namely, learning how to prove something that isn’t already known).

We don’t expect you to reprove arithmetic. Anything from high school math can be used without proof. [[[Tom: Presumably this doesn’t mean in the first assignment, where Peano axioms are being exercised.]]]

Once something has been proved, you can use it again without proof, citing the theorem. Once it has been used so often as to become commonplace, you can even stop citing the theorem.

No new axioms!

When in doubt about whether something can be stated without proof, prove it!

Elements of a proof:

• correctness

• clarity

• brevity

• elegance

“The book proof” (Paul Erdos).

[[[David listed the above characteristics. I would add some more:]]]

A good proof:

• Has nice structure, like a good program. It is broken up into separate “lemmas” that prove key intermediate properties. This makes it easy to understand the reasons why the whole thing works. It also makes it more likely that pieces can be reused.

• Has its pieces presented abstractly and generally. If you give an intermediate fact, state it as generally as you can. This avoids cluttering up a key piece of a proof with unnecessary hypotheses and arguments. It also makes it more likely you can reuse the piece.

16 Lecture 1: Notes 1 This is exactly analogous to what you do in designing subroutines for programs – make them general and abstract.