Massachusetts Institute of Technology Lecture 5 6.042J/18.062J: Mathematics for Computer Science February 15, 2000 Professors David Karger and Nancy Lynch

Notes 5

1 Induction and Recursive Algorithms

We’ve spent a lot of time studying induction. This is because induction comes up all the time in analyzing computation. Why? Because induction is a “one step at a time” proof method. Computations also evolve “one step at a time.” In fact, many of our inductive proofs can be seen as algorithms for solving a problem.

• We proved any 2^{n} by 2^{n} board can be tiled with L shapes.

If you look at the inductive proof, we actually gave a procedure for carrying out that tiling. Break the board in four, tile each piece, and merge the tilings. What kind of algorithm is this? recursive: it calls itself on smaller subproblems.

• The induction for existence of a fair ordering in a tournament gives a recursive algorithm for finding one.

• The definition of the Fibonacci numbers is also expressing a recursive algorithm for computing them.

• And the inductive proof for Nim gave a game-winning algorithm.

Just about any inductive proof that a problem has a solution is in fact providing a recursive algorithm for solving the problem. Conversely, when you have a recursive algorithm that you think solves some problem, induction is the usual way to prove that it works. In fact, an inductive proof can be thought of as a recursive algorithm for writing down a proof:

Procedure Write-Proof-For(n)

if n is 0

Write-the-base-case else

Write-Proof-For(n − 1) Write-Inductive-Step(n)

Assuming we have rule for writing down the base case P (0), and writing down the inductive step P (n − 1) ⟹ P (n) for any n, we can write down a proof of P (k) recursively: we recursively prove P (k−1), then we use the inductive step proof to prove P (k−1) ⟹ P (k).

1.1 RSA

RSA is at the core of many of the secure communication protocols we use today (SSH, SSL, etc). RSA works by treating any given message (sequence of bits) as a big number x and applying a mathematical function that “hides” the number x unless you know a secret key.

More precisely, it computes the remainder r of x^{a} on division by b, where a and b are two specially constructed large numbers. It turns out that if someone is given r and knows the right secret about the origin of a and b, they can “invert” the exponentiation and get x.

So, how can we compute the exponentiation? The most obvious way is to work from the recursive definition:

• x^{0} = 1

• x^{n} = x · x^{n−1}

This is a definition of x^{n}, but also gives a recursive algorithm for computing it:

Procedure Exp(x, n)

if n = 0

return 1 else

return x ∗ Exp(x, n − 1)

Now it’s pretty “obvious”, but how would you prove that algorithm Exp returns the right answer? Well, it doesn’t always: what happens if you call Exp(5, −2)? Exp(8, 2.5)? (Trying to prove correctness can reveal implicit assumptions. We assume n is a natural number).

Theorem 1.1 For all x, n ∈ ℕ, a call to Exp(x, n) returns x^{n}

Note: the procedure Exp above can be thought of as (i) a computational process or (ii) a recursive function definition (defined as the output of the computational process). We need to prove that it defines the same function as x^{n} We are actually claiming two distinct things in the theorem:

• The algorithm returns something (i.e., doesn’t run forever)

• The algorithm returns the right something.

Proof. By induction on n. The base case, that Exp(x, 0) returns x^{0} = 1, follows by inspection: on that call, the “if” condition is satisfied, so we return 1. For the inductive step, assume Exp(x, n) returns x^{n} and prove for n + 1. Consider the call to Exp(x, n + 1). n+1 > 0, so the “else” clause is invoked. We call Exp(x, n) which, by assumption, returns x^{n}.

We multiply that quantity by x, yielding x^{n+1}, and return it. This proves that Exp(x, n + 1) returns the right value.

OK, we’ve shown the procedure returns the right answer. Is it a good implementation? How many multiplications are done to compute x^{n}? By induction, there are n multiplications. Remember those “128 bits keys” we aren’t allowed to export because cryptography is considered a weapon? That means the number n has about 128 bits. So we will be doing about 2^{128} multiplications. We’ll still be working on it when 6.042 finishes...

1.2 Fast Exponentiation

Let’s cut down on the number of multiplies. Suppose we want to compute x^{a} and a is even, say 2c. So we want to compute x^{a} = x^{2c} Any obvious way to beat a multiplies? Note x^{2c} = (x^{c})^{2} So compute x^{c} (c multiplies) and then square it (1 multiply). Total: c + 1 = 1 + a/2 multiplies. What if a is odd?

Generalize this:

Procedure FastExp(x, n)

if n = 0

return 1 else

if n is even let y = FastExp(x, n/2) return y^{2}

else

let y = FastExp(x, (n − 1)/2)
return x ∗ y^{2}

Theorem 1.2 For all x, n ∈ ℕ, a call to FastExp(x, n) returns x^{n}

Proof. Induction again. Base case (n = 0) follows by inspection as before. So let’s prove the inductive step. What kinds of subproblems are we going to use to prove it works for n? not just n − 1, so let’s use strong induction.

Suppose the theorem is true for every k < n, let’s prove it for n Suppose we call FastExp(x, n). We can assume n ≠ 0 (we already did a proof for 0) so the else clause is selected in the algorithm. Two cases. If n is even (say n = 2m) then we call FastExp(x, m). By strong induction this returns x^{m}, which we assign to y. Then we return y = (x^{m})^{2} = x^{n} as required, so the proof works for this case. If n is odd (say n = 2m + 1) then we call FastExp(x, m) and set y to the result. Then we return x ∗ y^{2} = x ∗ (x^{m})^{2} = x^{2m+1} = x^{n} So the proof works in this case too. These are all the cases. QED.

How many multiplications does this algorithm do?

Theorem 1.3 If n has b bits, then FastExp(x, n) performs at most 2b multiplications.

Proof. Induction on b. Base case: If b = 0, then n = 0 and we return without doing any multiplications. So the base case is true. Inductive step: assuming it true for b bits, consider a b + 1-bit number n. We execute a recursive call to FastExp on either n/2 or (n − 1)/2, depending whether n is odd or even. Both of these numbers have at most b bits. So by induction, the recursion does at most 2b multiplications. Then we do either one more (even case) or two more (odd case). So at most two more. Overall, at most 2b + 2 = 2(b + 1) multiplications. QED.

Using this algorithm, a 128-bit key means only 256 multiplications: quite an improvement!

1.3 Greatest Common Divisor

Given 2 positive natural numbers, a, b ∈ N^{+}, their greatest common divisor is the largest number that divides both without remainder.

(Recall notation: a | b means a divides b.)

Finding GCD is a problem that must be solved to generate the “secret” used to decrypt RSA messages.

One of the first recorded algorithms—Euclid’s algorithm—finds greatest common divisors. Key lemma for “simplifying” problem:

Lemma 1.4 Given x > y, let r = x − y. Then any common divisor of x and y is also a common divisor of y and r, and vice versa.

Corollary 1.5 GCD(x, y) =GCD(y, r)

Proof. (of the corollary). (x, y) and (y, r) have exactly the same set of divisors (by lemma). Thus, the biggest number in each set, namely the greatest common divisor, is the same.

Proof. (of the lemma). Suppose d | x and d | y. So x = zd and y = wd. Thus r = x − y = d(z − w) so d | r, meaning d is a common divisor of y (by assumption) and r (by deduction). The other direction of proof is similar.

From this lemma we get Euclid’s algorithm:

Procedure Euclid(x, y)

if x = 0 return y else if y = 0 return x else if x > y return Euclid(y, x − y) else return Euclid(x, y − x))

Example: Find GCD(112, 84) = ? Step 1: 112 − 84 = 28, reduce to (28, 84). Step 2: 84 − 28 = 56, reduce to (28, 56). Step 3: 56 − 28 = 28, reduce to (28, 28). Step 4: 28 − 28 = 0, reduce to 28, 0 Now done, answer is 28.

Let’s prove this algorithm does the right thing. First faint worry: maybe we’ve got an infinite recursion? First thing to prove for any algorithm: it terminates

Lemma 1.6 For any natural number inputs x and y, Euclid(x, y) returns a value.

Why do we think this is true intuitively? Because the arguments “shrink” on each recursive call and eventually reach 0. Induction lets us formalize this.

Proof. Induction on x + y. More formally, we prove by strong induction on n that for all inputs x and y such that x + y = n, the algorithm terminates. Base case: if x + y = 0, then both x and y are 0 so we return immediately. Inductive step. Assume the claim for x + y < n and prove it for x + y = n. Now consider two cases. Either x ≥ y or y ≥ x. Suppose first that x ≥ y. So consider two subcases. If y = 0, then we return immediately and the inductive step is proved. If y > 0, then we make a recursive call with arguments y and x − y < x − 0 = x. So the sum of the two new arguments is less than x + y = n So the inductive hypothesis holds: the recursive call returns a value. So our original call returns (the same) value.

What about the second case (y ≥ x)? Well, I would have to write the same proof all over again, swapping x and y everywhere. To avoid such tedium, we say “without loss of generality, assume x ≥ y. This is code for “the other case works exactly the same way” and completes the proof.

Lemma 1.7 For any natural number inputs x and y, Euclid(x, y) returns GCD(x, y)

Proof. Induction on x+y as before. The base case is trivial (assuming we define GCD(0, 0) to be 0, which seems fair).

For the inductive step where x + y = n > 0, assume without loss of generality that x ≥ y. Then we return the result of a recursive call on Euclid(y, x − y). This means (by strong induction) that we return GCD(y, x − y). But we proved a lemma above that GCD(x, y) =GCD(y, x − y) so we are returning the right value.

Wait a minute, what’s with the GCD(0, 0) definition? Well, technically every number is a common divisor of 0 and 0, so it should be infinite?

Why doesn’t this matter? Because we never actually make a recursive call to (0, 0)—once one number is 0, we terminate.

So even though we still need a base case for the induction pattern, we never actually apply that base case. Instead our base case is implicit in the “subcase” of the inductive step where on argument is 0.

2 Sets

Today we give an overview of set theory, including relations on sets. Probably much of the first part of this will be review from high school math.

We will define an interesting kind of relation called an equivalence relation.

An important example of equivalence relation is number equivalence mod m, where m is a positive integer. We will define this relation and give some of its properties. We will conclude with an application of modular equivalence ideas to get fast RSA encryption and decryption.

2.1 Definitions

A set is a collection of objects. (Collection, object are undefined terms.)

The order of elements is not important. I.e., the sets {1, 2} and {2, 1} are considered the same set. Each element is included only once. So, if we write {a, b} or {a, a, b} they both denote the same set and the element a is considered to be present in the set only once. (repeating an element in the description of a set may be useful sometimes).

Examples of sets:

• {1, 2, 3}.

A simple finite set, obtained by enumerating all the elements.

• ∅, the empty set.

A special case of a finite set, with no elements.

• {∅}.

This funny set is different from ∅. It has one element, which is a set.

• {1, {1, 2}}.

This one contains two elements, one of which is a set itself.

The set happens to contain the other element 1, but that does not contradict the condition that an element is included only once.

• {n ∈ ℕ | n is even }.

An infinite set, specified by giving a universal set (domain of discourse) and a property. In general, given a domain of discourse D and a predicate P over D, {x ∈ D | P (x)} denotes the set of all x in D that satisfy predicate P.

We write that a ∈ A (“a is in A” or “a is an element of A”) if a is an object in the set A.

The empty set satisfies the proposition (∀x)x ∉ ∅.

Sets can be defined by axioms and deduction rules, giving a very formal axiomatic set theory. But usually, people reason about sets less formally, using naive set theory. Either way, set theory can be used as the basis for defining almost everything else in mathematics and is therefore regarded as the most fundamental part of math.

E.g., natural numbers can be defined as certain kinds of sets (!). For example we can represent the number 0 by the empty set ∅, number 1 by the set {∅}, number 2 by {∅, {∅}}, and so on. In general, if S_{n} is the set representing the number n, the successor of n can be represented by the set S_{n+1} = {∅} ∪ {{x} | x ∈ S_{n}}. Then the properties natural numbers can be proved as theorems of set theory, rather than being assumed.

Some typical axioms of axiomatic set theory: (Seem ridiculously obvious but aren’t)

• The axiom of extensionality says that two sets are equal iff they have the same elements.

E.g., the two sets {n ∈ ℕ | 2 divides n} and {2n | n ∈ ℕ}

are defined in different ways.

But they are “equal” as sets because they have exactly the same elements.

• The axiom of comprehension says that we can form a new set by including all objects of a given set that share a common property.

For example, {x ∈ ℝ | P (x)} where P (x) = “x is rational” denotes the set of all rational numbers. The axioms says that this is a set.

The set building notation that we used above has given the universe of discourse explicitly. Sometimes, when this seems obvious, we will leave this out. Just like what we did for predicate quantifiers. E.g., we might write: {n | n is even } if the universe ℕ seems clear. (Could be confusing – the universe could alternatively be the integers, positive and negative.)

Definition 2.1 We say that A ⊆ B (“A is a subset of B” or “A is contained in B”) if every element of A is also an element of B (that is, (∀x)(x ∈ A ⟹ x ∈ B)).

Definition 2.2 We say that A is a proper subset of B (A ⊂ B) if A ⊆ B and A ≠ B.

Note (∀X)∅ ⊆ X, i.e. the empty set is contained in any other set.

Q: Can we have both A ∈ B and A ⊆ B?

A: Yes, consider {1, {1}}.

Theorem 2.3 A = B if and only if A ⊆ B and B ⊆ A.

Proof by axiom of extensionality. Ignore here.

This observation is very important as it provides a simple way to prove that 2 sets are equal.

Example 1 Prove that the two sets {n ∈ ℕ | 2 divides n} and {2n | n ∈ ℕ} are equal.

Must show containment in both ways. ⊆: We assume x ∈ ℕ and 2 divides n, and show that there is some y ∈ ℕ such that x = 2y. This is true by definition of divisibility. ⊇: Assume x = 2y. Then x is divisible by 2, again by definition of divisibility.

Example 2 Slightly more interestingly:

The set of evens ≥ 2, i.e., {2n | n ∈ ℕ, n ≥ 1}, is equal to {x ∈ ℕ | ∃y, z odd , x = y + z}. ⊆: If x = 2n, then write x = 1 + (x − 1), sum of two odds. ⊇: If x = y + z, where both y and z are odd, then write y = 2a + 1, z = 2b + 1. Then x = 2(a + b + 1), which is even.

The size of a set A (the number of elements it contains) is written |A|. For example, the set {a, b, c} has size |{a, b, c}| = 3. The empty set has size |∅| = 0. The size of a finite set is a natural number. If A is an infinite set, we write |A| = ∞.

Notice that for any two finite sets A, B, we have |A∪B| ≤ |A|+|B| and equality holds if and only if A ∩ B = ∅. If we allow A and B to be infinite sets, we still have |A ∪ B| ≤ |A| + |B| if we define x + ∞ = ∞ + x = ∞ and x ≤ ∞ for all x ∈ ℕ and x = ∞.

2.2 Russell’s paradox

We might think that set theory is obvious, but this is not the case. Consider the set S = {x | x is a set}. This seems like a perfectly reasonable set (well, maybe a bit strange—note S ∈ S). If our set theory allows for such an entity, we get into trouble as follows.

Consider the set Q = {x ∈ S | x is a set and x ∉ x}.

If S is a set, then so is Q by standard axioms like comprehension. Note Q ⊆ S.

Then some sets are in Q and some aren’t.

Example of something in Q: The set A = {1, 2, 3} is not an element of itself, so A ∈ Q.

Example of something that is not in Q: The set P = { all infinite sets} is a set (by comprehension from S). Therefore, P ∈ P as the set P itself is infinite. So P ∉ Q.

The troublesome question is: is Q ∈ Q?

1. If Q ∈ Q then Q ∉ Q, a contradiction.

2. If Q ∉ Q then Q ∈ Q also a contradiction.

10 Lecture 5: Notes 5

This is clearly a paradox. In fact, this paradox, at the time it was discovered by Russell (∼ 1900), was a blow to the then current attempts to formalize all of mathematics axiomatically.

Mathematicians had to work very hard to find a consistent set of axioms (note that a consistent set of axioms does not allow this paradox to arise).

The problem with naive set theory is that you can form a set by describing its members, as we did for Q above. Axiomatic set theory has a specific set of rules for how you can form sets—rules that don’t allow the set Q above to be specified, and therefore avoid the paradox.

Fortunately, we’ll be working naively...

2.3 Basic set operations

There are several standard operations on sets. In axiomatic set theory, the claims that these operations yield new sets are axioms.

1. Union A ∪ B = {x | x ∈ A or x ∈ B}.

2. Intersection A ∩ B = {x | x ∈ A and x ∈ B}.

3. Difference A − B = {x | x ∈ A and x ∉ B}. Also written as A \ B.

4. Complement A^{c} = {x | x ∉ A}. Also written as Ā.

5. Symmetric difference A ⊕ B = A ∪ B − A ∩ B.

Sets and operations on sets can be represented using Venn diagrams (see textbook, sections 1.4, 1.5).

Union and intersection can be extended to apply to more than two sets. Given a finite collection of sets A_{i}, i = 1, …, n,

⋃_{i=1}^{n} A_{i}

is the set of elements in at least one of the A_{i}, and

⋂_{i=1}^{n} A_{i}

is the set of elements in all of the A_{i}. Both of these make sense if i = 1 – reduce to just A_{1}. In fact, for i = 0, default says union of no sets is ∅ and intersection of no sets is universal set. This definition of union and intersection of no sets might seem arbitrary, but in fact it can be motivated in several ways. For example, if we want to define the union of a finite family of sets A_{1}, A_{2}, …, A_{n}, we can define ⋃_{i=1}^{1} A_{1} = A_{1} and ⋃_{i=1}^{n+1} A_{i} = (⋃_{i=1}^{n} A_{i})∪A_{n+1}. If we want this recursive definition to work also for base case n = 0 we must set ⋃ ∅ = ⋃_{i=1}^{0} A_{i} = ∅. Analogously, for intersection. The recursive definition of ⋂_{i=1}^{n+1} A_{i} = (⋂_{i=1}^{n} A_{i}) ∩ A_{n+1} must have base case ⋂ ∅ = ⋂_{i=1}^{0} A_{i} = U if we want ⋂{A_{1}} = (⋂ ∅) ∩ A_{1} = A_{1}.

Instead of using explicit index, can just consider a set of sets: If S is a set of sets, then

⋃ S

denotes the union of all the sets in S, in any order. (Can prove that the order doesn’t matter, will do below.) Likewise,

⋂ S

denotes the intersection of all the sets in S.

Complement is different from all the other operations, because it assumes some universe U from which we extract the things not in A. All sets in question are supposed to be subsets of U. Given a universal set U, the complement operation can be expressed as A^{c} = U − A. The universe U is often implicit, where there seems to be little chance of confusion.

There is a close correspondence between set operations and logical operations: consider sets A, B and C defined by A = {x | P (x)}, B = {x | Q(x)}, C = {x | R(x)}, where P, Q, R are predicates.

• set union vs. predicate or: A ∪ B = C if and only if P (x) ∨ Q(x) ≡ R(x)

• set intersection vs. predicate and: A ∩ B = C if and only if P (x) ∧ Q(x) ≡ R(x)

• set difference vs. predicate “non implication”: A − B = C if and only if ¬(P (x) → Q(x)) ≡ R(x)

• set complement vs. predicate not: A^{c} = B if and only if ¬P (x) ≡ Q(x)

• set symmetric difference vs. predicate exclusive or: A ⊕ B = C if and only if P (x) ⊕ Q(x) ≡ R(x).

Predicates and sets often form alternative ways of looking at the same notions: Given predicates p and q, define P = {x | p(x)}, Q = {x | q(x)}. Then P ∪ Q = {x | p(x) ∨ q(x)}. Etc.

2.4 Power set (set of all subsets)

Another very important operation on sets is that of the power set of a given set A. It is denoted by P(A) and consists of the set of all subsets of A. In other words, P(A) = {x | x ⊆ A}.

In particular, for any set A, ∅ ∈ P(A) and A ∈ P(A). Ex: If A = {1, 2} then P(A) = {∅, {1}, {2}, {1, 2}}.

Theorem 2.4 If A has n elements, then P (A) has 2^{n} elements.

Proof. Ordinary Induction on n. BASE: n = 0. Then P (A) has one element, ∅. INDUCTIVE: Assume for n, prove for n + 1. Pick one element, a. The subsets can be divided into two groups – those containing a and those not containing a. Moreover, the set of subsets of A not containing a is P(A − {a}), and the set of subsets of A containing a is the set {{a} ∪ B | B ∈ P(A − {a}) which has the same number of elements as P(A−{a}). By induction hypothesis P(A−{a}) has 2^{n−1} elements, so the total is 2^{n−1} + 2^{n−1} = 2^{n}.

Q: I did a base case of 0 here. Does the inductive step from 0 to 1 work correctly?

A: Yes, it corresponds to the sets {∅} and {a}.

Because of this cardinality relationship, P(A) is sometimes denoted by 2^{A}.

2.5 Set identities

There are a large number of set identities involving relationships among sets built using various operations.

See p. 50 of book.

Here we just list some sample properties, and show how they can be proved.

Theorem 2.5 (Distributivity: Union distributes over intersection) A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)

Proof. We show the two sets are equal by proving that each is contained in the other.

1. A ∪ (B ∩ C) ⊆ (A ∪ B) ∩ (A ∪ C).

Suppose x ∈ A ∪ (B ∩ C). Then by definition, x ∈ A or x ∈ (B ∩ C). We argue by cases.

If x ∈ A, then by definition x ∈ A ∪ B and similarly x ∈ A ∪ C. It follows by definition of intersection that x is in the intersection of these two sets, which is the RHS.

On the other hand, if x ∈ (B ∩ C), then x ∈ B and x ∈ C. Then by definition of union, x ∈ A ∪ B and x ∈ A ∪ C. It again follows that x is in the intersection.

2. A ∪ (B ∩ C) ⊇ (A ∪ B) ∩ (A ∪ C).

Suppose x ∈ (A ∪ B) ∩ (A ∪ C). It follows that x ∈ A ∪ B and x ∈ A ∪ C. We argue by cases.

If x ∈ A, then certainly x ∈ A ∪ (B ∩ C).

If x ∉ A, then since x ∈ A ∪ B, we know x ∈ B. Similarly, x ∈ C. Therefore, x ∈ B ∩ C. It follows that x ∈ A ∪ (B ∩ C).

Theorem 2.6 De Morgan’s rule: (A ∪ B)^{c} = A^{c} ∩ B^{c}.

Proof. We have to show that (A ∪ B)^{c} ⊆ A^{c} ∩ B^{c} and that A^{c} ∩ B^{c} ⊆ (A ∪ B)^{c}.

1. (A ∪ B)^{c} ⊆ A^{c} ∩ B^{c}.

Let x ∈ (A ∪ B)^{c}. Then by definition of complement, x ∉ A ∪ B. It follows (proof by contradiction) that x ∉ A and x ∉ B. Therefore, again by definition of complement, x ∈ A^{c} and x ∈ B^{c}. Then by definition of intersection, x ∈ A^{c} ∩ B^{c}.

2. A^{c} ∩ B^{c} ⊆ (A ∪ B)^{c}.

Let x ∈ A^{c} ∩ B^{c}.

Then x is in A^{c} and in B^{c}.

Therefore, x is in neither A nor B and therefore not in A ∪ B.

We conclude that x ∈ (A ∪ B)^{c}.

The above two theorems have dual theorems produced by swapping ∩ and ∪. This is a general phenomenon – look at p. 50, arranged in pairs. Duality can be applied to any theorem that is true for all sets. Each theorem in a pair can be obtained from the other by swapping the operations ∩ and ∪, and the sets ∅ and U. Notice that complement operations are not changed.

Theorem 2.7 A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)

Theorem 2.8 (A ∩ B)^{c} = A^{c} ∪ B^{c}.

2.6 Cartesian products

Another important set operation is the Cartesian product. For sets A and B define A × B = {(a, b) | a ∈ A and b ∈ B}. Here, (a, b) is considered an ordered pair so that (a, b) ≠ (b, a).

Example 3 If A = {0, 1} and B = {x, y} then A × B = {(0, x), (0, y), (1, x), (1, y)}.

Order is important! In the above example, (a, 0) ∉ A × B, though (a, 0) ∈ B × A.

We can generalize Cartesian products to apply to more than 2 sets and define A_{1}×⋯×A_{n} = ∏_{i=1}^{n} A_{i} = {(a_{1}, ⋯, a_{n}) | a_{i} ∈ A_{i} for 1 ≤ i ≤ n}. Also, we write A^{n} for ∏_{i=1}^{n} A.

Examples:

• {0, 1}^{n} is the set of all n-tuples of 0’s and 1’s. Or, length n strings of 0’s and 1’s.

• ℜ^{2} = ℜ × ℜ is the real plane.

3 Relations

A “relation” is a fundamental mathematical notion expressing a relationship between elements of sets. It’s an abstract notion useful in practice for modeling many different sorts of relationships. It’s the basis of the relational data base model, the standard data model for practical data processing systems.

3.1 Definitions

A binary relation from a set A to a set B is a subset R ⊆ A × B. A binary relation on a set A is a subset R ⊆ A^{2} (I.e., a relation from A to A). We often write a ∼_{R} b or aRb instead of (a, b) ∈ R.

We can also define a ternary relation on A as a subset R ⊆ A^{3} or, in general, an n-ary relation as a subset R ⊆ A^{n}, or R ⊆ A_{1} × A_{2} × ⋯ × A_{n} if the sets A_{i} are different.

Relations are used to model many different things:

1. The relation “is taking class” as a subset of {students at MIT} × {classes at MIT}. A relation from students to classes.

2. The relation “is living in the same room” as a subset of {students at MIT}×{students at MIT}. A relation on students.

3. The relation “can drive from first to second city”. (Not necessarily directly – just some way, on some roads.)

4. Relation on computers, “are connected (directly) by a wire”

5. “meet one another on a given day”

6. “likes”

7. Let A = ℕ and define a ∼_{R} b iff a ≤ b.

8. Let A = P(ℕ) and define a ∼_{R} b iff |a ∩ b| is finite.

9. Let A = ℜ^{2} and define a ∼_{R} b iff d(a, b) = 1.

10. Let A = P({1, ⋯, n}) and let a ∼_{R} b if a ⊆ b.

11. In a tournament, the “who beats who” relation.

12. “Implies” is a relation on the set of boolean formulae.

3.2 Properties of Relations

There are several standard properties that are satisfied by (some) relations. These occur commonly enough that they are worth identifying. Sometimes interesting things can be deduced just from the abstract properties.

A binary relation R on A is:

16 Lecture 5: Notes 5

1. reflexive if for every a ∈ A, a ∼_{R} a.

2. irreflexive if for every a ∈ A, a ≁_{R} a.

3. symmetric if for every a, b ∈ A, a ∼_{R} b implies b ∼_{R} a.

4. antisymmetric if for every a, b ∈ A, a ∼_{R} b and b ∼_{R} a implies a = b.

5. transitive if for every a, b, c ∈ A, a ∼_{R} b and b ∼_{R} c implies a ∼_{R} c.

6. asymmetric if for every a, b ∈ A, a ∼_{R} b implies ¬(b ∼_{R} a).

There’s no sense to the specific negative prefixes; they just need to be remembered. The difference between antisymmetric and asymmetric relations, is that antisymmetric relations may contain pairs (a, a), i.e., elements can be in relations with themselves, while in an asymmetric relation this is not allowed. Clearly, any asymmetric relation is also antisymmetric, but not vice versa. Among our examples:

• Relation 2 is reflexive, symmetric, transitive.

• Relation 3 is reflexive, transitive.

Not necessarily symmetric, since roads could be one-way (Cambridge story), but in actuality.... But definitely not antisymmetric.

• Relation 4 is symmetric but not transitive.

Whether it is reflexive is open to interpretation.

• Relation 5 likewise.

• Relation 6 is (unfortunately) not symmetric. Not antisymmetric. Not transitive. Not even reflexive!

• Relation 7 is reflexive, antisymmetric, transitive.

• Relation 8 is not reflexive. It is symmetric. Not transitive.

Evens∩Odds is finite (empty), but not Evens∩Evens.

• Relation 9 is only symmetric.

• Relation 10 is reflexive, antisymmetric and transitive.

3.3 Representation

There are different ways of representing relations, e.g., for handling in a computer program.

We can describe by properties, as we did above. That’s about all we can do, for infinite sets. But for finite sets, usually we use some method that explicitly enumerates all the elements of the relation.

Some alternatives: Lists, Matrices, Graphs.

3.3.1 Lists

To represent a finite relation from set A to set B, just list all the pairs.

Ex: Consider for example the relation from A = {0, 1, 2, 3} to {a, b, c} defined by the list {(0, a), (0, c), (1, c), (2, b), (1, a)}.

Ex: The divisibility relation on natural numbers {1, ⋯, 12} is represented by the list: {(1, 1), (1, 2), …, (1, 12), (2, 2), (2, 4), …, (2, 12), (3, 3), (3, 6), (3, 9), (3, 12), (4, 4), (4, 8), (4, 12), (5, 5), (6, 6), (6, 12), (7, 7), (8, 8), (9, 9), (10, 10), (11, 11), (12, 12)}.

A more compact representations lists the “partners” for each item: 1 : 1, 2, 3, 4, … 2 : 2, 4, 6, 8, … 3 : 3, 6, 9, 12 etc.

Reflexivity in this representation: Contains all pairs (a, a). Symmetry: If contains (a, b) then contains (b, a). Transitivity: If contains (a, b) and (b, c) then contains (a, c).

3.3.2 Boolean matrices

Used for finite sets A, B Rows for elements of A, columns for B. 1 in position if the pair is in the relation, 0 otherwise.

Ex: The relation from {0, 1, 2, 3} to {a, b, c} of our first example is represented by the matrix

\begin{aligned}& a & b & c \\ 0 & 1 & 0 & 1 \\ 1 & 1 & 0 & 1 \\ 2 & 0 & 1 & 0 \\ 3 & 0 & 0 & 0 \end{aligned}

Ex: The divisibility relation over {1, 2, …, 12} is represented by the matrix

1 2 3 4 5 6 7 8 9 10 11 12
1 1 1 1 1 1 1 1 1 1 1 1 1
2 0 1 0 1 0 1 0 1 0 1 0 1
3 0 0 1 0 0 1 0 0 1 0 0 1
4 0 0 0 1 0 0 0 1 0 0 0 1
5 0 0 0 0 1 0 0 0 0 1 0 0
6 0 0 0 0 0 1 0 0 0 0 0 1
7 0 0 0 0 0 0 1 0 0 0 0 0
8 0 0 0 0 0 0 0 1 0 0 0 0
9 0 0 0 0 0 0 0 0 1 0 0 0
10 0 0 0 0 0 0 0 0 0 1 0 0
11 0 0 0 0 0 0 0 0 0 0 1 0
12 0 0 0 0 0 0 0 0 0 0 0 1

Reflexivity in this representation: major diagonal is all 1 Symmetry: the matrix is symmetric around major diagonal Transitivity: ...

3.3.3 Digraphs

Digraphs are ways to make a picture of a relation. The picture can reveal lots of “shape” (topological) properties of the relation that are not clear from the other representations (the matrix is good for algebraic properties).

The real power of graphs is for relations on a single set A. To represent a relation on A as a digraph we draw a dot (vertex) for each element of A, and draw an arrow from first element to second element of each pair in the relation. The digraph may contain self-loops, i.e. arrows from a dot to itself, associated to the elements a such that (a, a) is in the relation.

For relations from A to B ≠ A, you use two “columns” of vertices: A on left, B on right, and draw arrows from A vertices to B vertices. But we will focus on relations on A.

The divisibility relation over {1, 2, …, 12} is represented by the digraph Reflexivity: All nodes have self-loops. Symmetry: all edges are bidirectional (in which case, instead of drawing lots of bidirectional arrows, we just draw a line with no arrowhead). Transitivity: Short-circuits, for any sequence of consecutive arrows, there is a single arrow from the first to the last node.

We will devote time later to the topological properties of graphs. For now, just a way to represent relations.