Massachusetts Institute of Technology Lecture 8 6.042J/18.062J: Mathematics for Computer Science October 1, 1996 Professors Nancy Lynch and Tom Leighton

Notes 8

1 Partial Orders

Doing series of math structures: natural numbers and integers, sets, relations (including equivalence relns), functions.

Next kind of structure: partial order. A particular type of binary relation.

Important in computer science. Applications to task scheduling, Database concurrency control, Logical time in distributed computing

1.1 Definitions

1.1.1 Partial Order Definition

Definition 1.1 A binary relation R ⊆ A × A is a partial order if it is reflexive, transitive, and antisymmetric.

Quick review:

• reflexive: aRa

• transitive: aRb ∧ bRc ⇒ aRc

• symmetric: aRb ⇒ bRa

• antisymmetric: aRb ∧ bRa ⇒ a = b. i.e., ∀a ≠ b, aRb ⇒ ¬bRa. Means NEVER symmetric!

For comparison: Recall that a relation that is reflexive, transitive, and symmetric is an equivalence relation.

Examples of partial orders:

• A = ℕ, R =≤, easy to check reflexive, transitive, antisymmetric

• A = ℕ, R =≥, same.

• A = ℕ, R =<, NOT because not reflexive

• A = ℕ, R = | (divides), easy to check reflexive, transitive, antisymmetric

• A = P (ℕ), R =⊆, check reflexive: S ⊆ S, transitive: S ⊆ S′ ∧ S′ ⊆ S′′ ⇒ S ⊆ S′′, antisymmetric: S ⊆ S′ ∧ S′ ⊆ S ⇒ S = S′.

• A = “set of all computers in the world”, R = “is (directly or indirectly) connected to”. NOT a p.o. because it is not true that aRb ∧ bRa ⇒ a = b. In fact, it is symmetric, transitive. Equivalence relation.

• A = “set of all propositions”, R =⇒, NOT because not antisymmetric. Not symmetric either, so not equivalence relation.

• A = some set of tasks, R = “must be done before”, according to some task specification. Antisymmetry: Does aRb ∧ bRa ⇒ a = b? Hopefully, because otherwise a, b can never be done! So we will only deal with task specifications with this property. Transitive. BUT not reflexive. If we want to treat this as a partial order, we must make a small modification of R to “either = or must be done before”.

1.1.2 Reflexive and Transitive Closures

The construction we just did is called “taking the reflexive closure”. In general, if we have a binary relation R that is antisymmetric and transitive (but not necessarily reflexive), we can turn it into a partial order simply by adding in all pairs of the form (a, a). Adding in these pairs does not mess up the antisymmetry or transitivity properties. (Check this.)

A related construction is “taking the transitive closure”. It starts with a relation R that is antisymmetric and reflexive, but not necessarily transitive. However, it has no “cycles”, i.e., no instances where (a_{1}, a_{2}) ∈ R, (a_{2}, a_{3}) ∈ R, ... (a_{k−1}, a_{k}) ∈ R, and (a_{k}, a_{1}) ∈ R, where each pair consists of distinct elements, and where k ≥ 2.

E.g., “S is a subset of S’, with either the same number of elements (so S = S′) or else exactly one fewer”. This is reflexive, antisymmetric, not transitive. But no cycles.

Then add in all the transitively-determined pairs. What are these? Well, if (a, b) and (b, c) are in R, then must add in (a, c). But not enough just to do this for the original pairs. Must repeat, e.g., if (c, d) ∈ R, must now add in (a, d), etc. In general, must add in any pair (a_{1}, a_{k}) such that all the pairs (a_{1}, a_{2}), (a_{2}, a_{3}), ..., (a_{k−1}, a_{k}) are in R. Claim this is enough to make the result transitive. (For any two pairs assembled in this way that match up, the transitive combination is also included.) Moreover, adding all these pairs doesn’t mess up reflexivity (obviously) or antisymmetry (because the original relation had no cycles). So the result is a partial order.

E.g., in the subset example above, transitive closure adds in all subset pairs without any restriction on the number of elements in the set difference.

Could also start with not-necessarily reflexive, but antisymmetric and cycle-free, and take the “reflexive transitive closure” to turn it into a p.o. Just do both of the above constructions.

E.g., “S is a subset of S’, with S missing exactly one element from S′.

1.1.3 No Cycles

In taking the transitive closure, we had to start with a relation without cycles. In fact, it turns out that every partial order is cycle-free:

Theorem 1.2 A partial order has no cycles of length ≥ 2, i.e., no sequences of pairs of the form (a_{1}, a_{2}), (a_{2}, a_{3}), ...(a_{k−1}, a_{k}), and (a_{k}, a_{1}), k ≥ 2, all in R, each pair consisting of distinct elements.

Proof. By contradiction. Assume that such a cycle exists in R. Then by transitivity (maybe applied several times), (a_{1}, a_{k}) ∈ R. Since also (a_{k}, a_{1}) ∈ R, antisymmetry implies that a_{1} = a_{k}, which is a contradiction.

1.1.4 Posets and Hasse Diagrams

The reflexivity, antisymmetry and transitivity properties are abstract properties that generally describe “ordering” relationships. So often write an ordering-style symbol like ⪯ instead of just a letter like R, for a p.o. relation. But this could be misleading – note that ≥ is a p.o. on natural numbers, as well as ≤. If we use the ⪯ symbol for ≥, things look really funny. (In cases like this, better to use R.) A partial order is always defined on some set A. The set together with the partial order is called a “poset”:

Definition 1.3 A set A together with a partial order ⪯ is called a poset (A, ⪯).

Representing posets:

Can usually be nicely drawn as a graph (Informally, dots and arrows, will do more formally next time.) Draw an arrow from a to b if aRb. (Similar to what we’ve already done with relations.)

E.g.: Draw a poset that describes the order in which one would put on clothes (men only here). The set is of clothes, and the partial order specifies what should be put on before what.

[figure: .; left sock; left shoe; right sock; shirt; underwear; tie; jacket; belt; pants; right shoe]

Notice that some arrows are left out. E.g., underwear precedes belt. But all those omitted follow from transitivity. Loops (which express reflexive relationships) also aren’t shown.

Definition 1.4 A Hasse diagram for a poset (A, ⪯) is a graph:

- whose vertices are the elements of A,

- whose edges (arrows) represent pairs (a, b), where a ≤ b but a ≠ b, - that contains no “transitive edges”,

- for which all pairs in ⪯ are obtainable by transitivity from edges in the diagram.

To convert the diagram above to an “official form” Hasse diagram, must remove the pants → jacket arrow. (Remove on board.)

Notice that the graph above has directed paths, that is, sequences of nodes a_{1} ⋯ a_{n} where (∀i < n)a_{i}Ra_{i+1}. However, it does not have any directed cycles, a directed path where a_{1} = a_{n}. Good thing, because otherwise we could get stuck trying to get dressed. As proved above, this is true of all posets.

1.1.5 Partial vs. Total Orders

A partial order is called partial because it is not necessary that an ordering exist between every pair of elements in the set.

E.g., Lshoe and Rshoe have no prescribed ordering between them.

E.g., For two sets, it’s not necessary that either be a subset of the other.

When there is no prescribed order between two elements we say that they are “incomparable”.

Definition 1.5 a and b are incomparable if neither a ⪯ b nor b ⪯ a.

Definition 1.6 a and b are comparable if a ⪯ b or b ⪯ a.

E.g., for subsets of ℕ, {1, 2, 3} and {2, 3, 4} are incomparable.

However, a partial order need not have incomparable elements. As a special case, we can have a p.o. in which there is a specified order between every pair of elements. Such partial orders are called “total orders”.

Definition 1.7 A poset (S, ⪯) is totally ordered if (∀a, b ∈ S)[a ⪯ b ∨ b ⪯ a].

Examples:

• S = ℕ, ⪯=≤

• S = ℕ, ⪯= |, NOT because neither 3|5 nor 5|3

• S = P (ℕ), ⪯=⊆, NOT because neither {3} ⊆ {5} nor {5} ⊆ {3} (just said this above)

• Lexicographic order on pairs of natural numbers, defined by: (a_{1}, a_{2}) ⪯ (b_{1}, b_{2}) if and only if either a_{1} < b_{1}, or else a_{1} = b_{1} and a_{2} ≤ b_{2}. Do some examples here. (2, 37) ⪯ (2, 38) ⪯ (3, 2). “Lexicographic” because like dictionary.

The Hasse diagram of a total order looks like a line.

This has been the basic material. Book has a bit more: minimal, maximal elements (nothing smaller, nothing larger) lower bounds, upper bounds (for a subset – ≤ everything in the subset, or ≥) lub, glb, lattices. I’ll leave you to read these parts, do some for HW.

Rest of the hour I’ll cover some slightly more advanced material and some applications in c.s.

1.2 Topological Sorting

Sometimes when we have a partial order, e.g., of tasks to be performed, we want to obtain a consistent total order, e.g., an order in which to perform all the tasks, one at a time, so as not to conflict with the precedence requirements.

The task of finding an ordering that is consistent with a partial order is known as “topological sorting”. (Why this name?)

Definition 1.8 A topological sort of a finite poset (A, ⪯) is a total ordering of all the elements of A, a_{1}, a_{2}, ⋯, a_{n} in such a way that ∀i < j, either a_{i} ⪯ a_{j} or a_{i} and a_{j} are incomparable.

For example, underwear, shirt, Lsock, Rsock, pants, tie, Lshoe, Rshoe, belt, jacket is a topological sort of how to put on a suit.

One of the nice facts about posets is that such an ordering always exists and is even easy to find:

Theorem 1.9 Every finite poset has a topological sort.

We’ll prove this by induction. On the number of elements in the domain of the poset. The basic idea is to pick a “smallest” element to start and then proceed recursively.

Definition 1.10 A minimal element a in a poset (A, ⪯) is one for which (∀b ∈ A)[a ⪯ b ∨ a and b are incomparable ].

Notice that there can be more than one minimal element. There are 4 in the clothes example: Lsock, Rsock, underwear, shirt.

Lemma 1.11 Every finite poset (A, ⪯) has a minimal element.

Proof. By contradiction. Assume no minimal elements. Then (∀a ∈ A)(∃b ≠ a)[b ⪯ a].

Start from any element of a and construct a sequence of successively smaller elements, by using the fact above. At each step, can get another element preceding the most recent. Can’t repeat any of the previous ones in the chain, since that would create a cycle (and we know a poset has no cycles). So just keep going forever, creating an infinite chain of distinct elements.

But this is nonsense, because A is a finite set. Contradiction, so there must be some minimal element.

Now we can prove the existence of the topological sort.

Proof. By (ordinary) induction. P (n) = any poset with n elements has a topological sort. Base case (P (1)): for a poset with one element the one element makes a topological sort.

Inductive step: Assume (P (n)). Consider a poset of n + 1 elements. By lemma it has a minimal element u. Consider the same relation on A − {u}. It is easy to check that it is still reflexive, transitive, and anti-symmetric. So by the inductive hypothesis, A − {u} has a topological sort. We put u first and now have a topological sort on the whole thing. (Say why – according to def of topological sort – u can’t be in the wrong order w.r.t. any of the later elements.)

Thus every poset has a topological sort, by induction.

E.g.: Dressing example: Construct ordering by picking one at a time. In each case, look at the remaining p.o. for remaining elements. Lsock, shirt, tie, Rsock, underwear, pants, Lshoe, belt, jacket, Rshoe

E.g., subsets of {1, 2, 3, 4} Draw lattice. ∅ is the unique minimal element, then have choices, e.g., do: {1}, {2}, {1, 2} {3}, {1, 3}, {2, 3}, {1, 2, 3}, {4}, {1, 4}, {2, 4}, {3.4}, {1, 2, 4}, {1, 3, 4}, {2, 3, 4}, {1, 2, 3, 4}

1.3 Parallel Task Scheduling

When elements of a poset are tasks that need to be done and the partial order is precedence constraints, topological sorting provides us with a legal way to execute tasks sequentially, i.e., without violating any precedence constraints. But what if we have the ability to execute more than one task at the same time? For example, say tasks are programs, partial order indicates data dependence, and we have a parallel machine with lots of processors instead of a sequential machine with only one. How should we schedule the tasks? Our goal should be to minimize the total time to complete all the tasks. (For simplicity, say all the tasks take the same amount of time and all the processors are identical.) So: Given a finite poset of tasks, how long does it take to do them all, in an optimal parallel schedule? Assume that all tasks take 1 unit of time and we have an unlimited number of processors.

Use partial order concepts to analyze this problem.

On clothes example, we could do all the minimal elements first (Lsock, Rsock, underwear, shirt), remove them and repeat. Need lots of hands, or maybe dressing servants. (So do pants and tie next, and then Lshoe, Rshoe, and belt, and finally jacket.)

Can’t do any better, because the sequence shirt, pants, belt, jacket must be done in that order.

A sequence like this is called a chain.

Definition 1.12 A chain in a poset is a sequence of elements of the domain, each of which is smaller than the next in the partial order (⪯ and ≠).

Clearly, parallel time ≥ length of any chain. This is true, because if we used less time, then by pigeonhole two tasks in the chain would have to be done at the same time, but by definition of chain this violates precedence constraints.

A longest chain is also known as a critical path.

So we need at least t steps, where t is the length of the longest chain. Fortunately, it is always possible to use only t:

Theorem 1.13 Given any finite poset (A, ⪯) for which the longest chain has length t, it is possible to partition A into t subsets, A_{1}, A_{2}, ⋯, A_{t} such that (∀i ∈ [1, t])(∀a ∈ A_{i})(∀b ⪯ a, b ≠ a)[b ∈ A_{1} ∪ A_{2} ∪ ⋯ ∪ A_{i−1}].

That is, can divide up the tasks into t “groups” so that for each group i, all tasks that have to precede tasks in i are in smaller-numbered groups.

Corollary 1.14 For (A, ⪯) and t as above, it is possible to schedule all tasks in t steps.

Proof. ∀i, schedule all elements of A_{i} at time i. This satisfies the precedence requirements, because all tasks that must precede a task are scheduled at preceding times.

Corollary 1.15 parallel time = length of longest chain

So it remains to prove the partition theorem:

Proof. Use the rule: Put each a in set A_{i}, where i is the length of the longest chain ending at a.

See this works: It gives just t sets, because the longest path is length t.

Need to show (∀i)(∀a ∈ A_{i})

(∀b ⪯ a, b ≠ q)[b ∈ A_{1} ∪ A_{2} ∪ ⋯ ∪ A_{i−1}].

Proof by contradiction: If b ∉ A_{1} ∪ A_{2} ∪ ⋯ ∪ A_{i−1} then there is a path of length > i − 1 ending in b, which means there is a path of length > i ending in a, which means a ∉ A_{i}. This is a contradiction.

So with an unlimited number of processors, the time to complete all the tasks is the length of the longest chain. For HW you’ll see what happens when there are limited processors.

It turns out that this theorem is good for more than parallel scheduling. It is usually stated as follows.

Corollary 1.16 If t is the length of the longest chain in a poset (A, ⪯) then A can be partitioned into t antichains.

Definition 1.17 An antichain is a set of incomparable elements.

This is a corollary because we showed before how to partition the tasks into sets that could be done at the same time, which means that these were incomparable. (If any two comparable then one would have to precede other and so couldn’t be done at same time.)

Review:

• chain: all elements comparable (total order)

• antichain: all elements incomparable

1.4 Dilworth’s Theorem

We can use this to prove a famous result about posets:

1.4.1 The Basics

Theorem 1.18 (Dilworth) ∀t, every poset with n elements must have a chain of size > t or an antichain of size ≥ \frac{n}{t}.

Ex: Dressing poset. n = 10. Try t = 3. Has chain of length 4. Try t = 4. Has no chain of length 5, but has antichain of size 4 ≥ \frac{10}{4}.

Proof. Assume there is no chain of length > t. So, longest chain has length ≤ t. Then by previous result, the n elements can be partitioned into ≤ t antichains. (Can assume exactly t, since could have some empty.) So by pigeonhole there is an antichain with ≥ \frac{n}{t} elements. As needed.

For instance:

Corollary 1.19 Every poset with n elements has a chain of length > \sqrt{n} or an antichain of size ≥ \sqrt{n}.

Set t = \sqrt{n} and this is immediate.

Posets arise in all sorts of contexts, and when they do Dilworth’s theorem can have interesting implications.

1.4.2 Increasing and Decreasing Sequences

A mathematical example: In any sequence of n different numbers, there is either an increasing subsequence of length > \sqrt{n} or a decreasing subsequence of length ≥ \sqrt{n}.

E.g., 6 4 7 9 1 2 5 3 8 has the decreasing sequence 6 4 1 and the increasing sequence 1 2 3 8. That is, it has both.

We can get this using Dilworth’s theorem; the trick is to define the appropriate poset.

The domain is the set of values in the sequence.

For the ordering, define a ⪯ b if either a = b, or else (a < b and a comes before b in the sequence).

Check this is reflexive (stated explicitly), transitive, and antisymmetric.

A chain corresponds to a sequence that increases in value and moves to the right, that is, an increasing sequence.

But what does an antichain correspond to?

Lemma 1.20 If a, b are incomparable and a > b then a precedes b in the sequence.

Proof. By contradiction. Assume b precedes a. Then b < a and b precedes a, so b ⪯ a in the partial order. This contradicts our assumption that a and b are incomparable.

Extend this to more than 2 elements:

Lemma 1.21 If a_{1}, a_{2}, ⋯, a_{t} are incomparable and a_{1} > a_{2} > ⋯ > a_{t} then a_{1}, a_{2}, ⋯ a_{t} form a decreasing subsequence.

Proof. (∀i), the fact that a_{i} > a_{i+1} implies that a_{i} precedes a_{i+1} in the sequence, by the previous lemma.

So given an antichain, arrange the elements so that they are in decreasing order and the “left of” relation follows, giving a decreasing subsequence.

Dilworth’s theorem implies that there is either a chain of size > \sqrt{n} or an antichain of size ≥ \sqrt{n}. By the analysis just above, this yields either an increasing sequence of length > \sqrt{n} or a decreasing subsequence of length ≥ \sqrt{n}.

1.4.3 Number of Comparisons for Database Setup and Searching

Consider a “database” of records with (distinct) numerical keys. Suppose when records are inserted, some comparisons of these keys occur, and the database maintains a record of these comparisons.

E.g., at one extreme, database could be kept totally sorted. E.g., totally unsorted.

Each comparison costs something – the time to perform it.

Now suppose that after this setup, a search is to be performed on the database to determine if a record with a particular key k appears anywhere in the database. The search is supposed to be based on comparisons only. Then the worst-case number of comparisons it takes depends on the database partial order.

E.g., if db is totally sorted, can compare with middle elements, then middle of halves, etc. Binary search. Approximately log n comparisons needed.

E.g., if db totally unsorted, then in the worst case, have to compare with every element.

This is because each comparison with an element in the db gives no information about whether the sought element might be any of the others in the db.

In fact, if there is an antichain of size k, then in the worst case, the new key could be any of the elements of the antichain. So in the worse case, have to compare with all k, since each one gives no information about the others.

See what Dilworth’s theorem says about the database partial order. For any t, either have a chain of length > t, or an antichain of size ≥ \frac{n}{t}. Setting up a chain of length > t requires at least t comparisons. The antichain requires at least \frac{n}{t} comparisons. So this says that we have to pay the cost sometime – tradeoff – either we do > t comparisons while setting up the db, or at leaast \frac{n}{t} in the worst case to search the db.

More refined analysis is possible. That’s because reliably setting up a long chain, say length t, in the database actually requires more than t comparisons. Because in the worst case, the comparisons won’t come out in the most desirable way, setting up a linear chain. Ex: Compare a and b, find out a > b. Then compare b and c. Would be nice if we found out b > c. But we might not – might get c > b. Then we don’t have a chain of length 3, but still an antichain of size 2. To get a chain, need an additional comparison.

Return to this in analysis of algorithms.

1.5 Logical Time in Distributed Systems

One last use of p.o. No use of powerful theorems here – just showing an important place where a p.o. is used as a mathematical model for a real computational situation.

A distributed computation consists of a sequence of “steps” occurring at each of a collection of n “processes”. In addition to performing local steps, the processes can send messages to each other. So, have send steps and receive steps.

The processes are mostly working independently. The only interaction is by means of the messages. So, the only dependence among steps (feeding data from one to another, using results to make decisions, etc.) is by:

1. Successive steps of same process.

2. Send vs. receive of same message.

Taking reflexive transitive closure leads to partial order describing the dependence among the computation steps.

Basic theorem: Any topological sort of this p.o. describes a possible sequence in which all the distributed steps could occur. (No changes in what happens based on reordering incomparable steps.)

Example of what one might do with this p.o.: Assign “logical time” to every step. Instead of an actual real time. The time is just any assignment of real numbers to the steps that is consistent with the p.o.: when one step precedes another in the p.o., it must have a smaller logical time.

Behaves like real time: Formally, can reorder the steps in logical time order, still consistent with the p.o. Then the system is behaving as if the logical time were actually real time.

Can be maintained by simple distributed algorithm. (Piggybacks time on message, adjusts local times in response to what is heard in some messages.)

Although logical time isn’t the same as actual real time, for many purposes, it can be used as an adequate substitute. Logical time can be used, e.g., to timestamp files to keep updated copies consistent.