Massachusetts Institute of Technology Lecture ?? 6.042J/18.062J: Mathematics for Computer Science October 8, 1996 Professors Nancy Lynch and Tom Leighton

Notes 10

This lecture finishes up graphs, does all of the next structure – trees, and then starts the last in the series of structures – state machines.

1 Graphs

1.1 Planar Graph Definitions and Examples

We will finish up the unit on graphs by discussing planar graphs, undirected graphs that can be drawn in the Euclidean plane without any lines crossing. Not exactly discrete math, since deals with Euclidean plane (reals, geometry).

Example: Mesh, cycle, star are planar

Example: K_{4}, complete graph of 4 nodes, planar Because can redraw in plane. Same graph, different representation.

Example: K_{5}, complete graph of 5 nodes, nonplanar Proof? Can do by case analysis – LTTR, but do another such proof below.

Example: K_{3,3}, complete bipartite graph, nonplanar Proof: Vertices a_{1}, a_{2}, a_{3} and b_{1}, b_{2}, b_{3}. Must have circuit a_{1}, b_{1}, a_{2}, b_{2}. Edges can’t cross, so they must split the plane into an inside and an outside. Consider where a_{3} is. Case 1: a_{3} inside. Then connects to b_{1} and b_{2}, splitting inside into two subregions. Consider where b_{3} might go. Outside: Then can’t connect to a_{3}. In one of the two subregions: Then can’t connect to the opposite a vertex. Contradiction.

Case 2: a_{3} outside. Similar. Connects to b_{1} and b_{2}, splitting outside into two subregions. Consider where b_{3} might go. Inside: Then can’t connect to a_{3}. In one of two subregions: Then can’t connect to the opposite a vertex.

Similar case analysis argument works to show that K_{5} isn’t planar.

Have already seen one fact about planar graphs, in recitation: Already know that planar graphs are all 4-colorable. Proof by computer analysis (after reducing the problem by clever analysis to consideration of about 2000 specific graphs).

Now give two other basic theorems about planar graphs: Euler’s formula Kuratowski’s theorem

1.2 Euler’s Formula

A connected planar graph divides the plane up into regions (counting the “outside” too as a region).

Example: Fig 1

Euler noticed a simple relationship between the numbers of regions, vertices, and edges in any connected planar graph.

In example: Number of regions: 7 Number of vertices: 11 Number of edges: 16

Invariant? Well, can always add another edge by adding a vertex. Suggests the invariant might involve e and v in a way that lets increases “balance”, e.g., the difference e − v might be part of the invariant. Also, can usually add another region by adding an edge (and no vertices). Suggests that r − e might be involved.

The correct formula turns out to be: r = e − v + 2. Here, 7 = 16 − 11 + 2.

Why is this always true?

Proof. By induction on the number of edges in the connected planar graph. BASE: e = 0. Connected graph with no edges is just a single vertex. Has r = 1, e = 0, v = 1, 1 = 0 − 1 + 2.

INDUCTIVE STEP: Suppose this is true for any connected planar graph with e edges. Now consider an arbitrary connected planar graph G with e + 1 edges.

We will find a good edge to try to remove, to obtain a graph with fewer edges to consider. Can’t just remove any edge, since it could disconnect the graph. So need a way to select a good edge to remove. To do this, start at any node and trace edges without retracing any. Keep doing this until you can’t continue. Let e_{l} be the last edge on this path. There are two possibilities:

1. e_{l} ends up at a node with no other incident edges in G.

In this case, remove e_{l} and its final endpoint.

What’s left, G′, is not disconnected – just removed a single edge that was jutting out. G′ has one fewer edge, so by inductive hypothesis, it satisfies the equation r = e−v+2. To get G, we add one edge and one vertex, while keeping number of regions unchanged. This means that the equation is still true after the addition.

2. e_{l} leads to a node with at least one other edge in G, but all such other edges have been traversed already.

In this case, e_{l} is part of a cycle in G.

So again removing the edge doesn’t disconnect the graph – just broke a cycle.

The reduced graph G′ has one fewer edge, so by inductive hypothesis, it satisfies the equation r = e − v + 2. This time, to get G, we add one region, and one edge, while keeping the number of vertices unchanged. So again, the equation remains true after the addition.

Example: Fig. 2:

1.3 Kuratowski’s Theorem

This is a nice (but not so easy to prove) result that says exactly which graphs are not planar.

Already know that K_{5} and K_{3,3} are not planar. Turns out those are essentially the only cases: K’s theorem says that any non-planar graph has one of these two embedded within it.

What do I mean by “embed”?

It means that there is some subgraph (that is, some graph obtained by taking subsets of vertices and edges) that is “like” K_{5} or K_{3,3} – the same, except that some edges may be divided into several edges, end-to-end. (Has fancy name: homeomorphic.)

Example: Fig. 3 – this is non-planar, because of the K_{5} embedded in it. Note it isn’t easy to find the embedded graph; in general, might have to try all possible matches.

Example: Fig. 4 – planar, no K_{5} or K_{3,3} embedded. How do we know? Can try all matches and fail. Or, give planar representation: ...

2 Trees

Important special type of graph for CS applications.

2.1 Definitions

Tree: Undirected, connected graph with no simple circuits (of length ≥ 1).

Draw example.

Forest: No circuits, but not necessarily connected.

Draw example.

In nontrivial trees, with more than one node, use terminology: Leaf – node with only one neighbor. Internal node – non-leaf

Alternative definition often given for tree: Unique simple path between every pair of vertices. Equivalent definition – same set of graphs defined in both cases. LTTR, in book p. 533.

Trees in CS (and elsewhere) are often drawn in a particular orientation, one node at the top, called the root, and the others on paths moving downward. (Of course, real trees, like maples, etc., have the root at the bottom, but no one in CS seems to have noticed this.)

Depends on distinguishing one node, any node, as root. Level 0 node. Then all its neighbors drawn just below it, level 1. Then all their neighbors, level 2, etc. Rooted tree = undirected tree + root. Maximum level number = height of rooted tree.

Sometimes “orient” (direct) all the edges downward, producing a directed tree. (Undirected) tree + root yields directed tree. Note different roots yield different directed trees (can even have different heights). Draw example, Fig. 5.

Could also change the order of siblings, same tree.

With rooted tree, use terminology parent, sibling, ancestor, descendent. From genealogy trees (but neglecting spouse edges). Draw one.

Def: subtree.

Definition: A rooted tree is an m-ary tree, m ≥ 1, if every internal vertex has no more than m children.

Special case of m = 2 is binary tree. Almost the same as we defined earlier in term (as an example to illustrate recursive definitions); there is a slight difference in that now we’re not distinguishing left and right. Sometimes we do this, but then, if we’re being careful, we call the tree an ordered rooted tree instead of just a rooted tree. Sometimes I’ll be sloppy and not explicitly say “ordered”, although I mean it.

Examples of uses of trees:

Representing organization structure (who reports to whom). Representing computer file systems (UNIX, almost)

Will see more in a few minutes.

2.2 Counting on Trees

Give some results about number of edges vs. vertices, relate to arity of tree, etc.

Theorem 2.1 Any tree with n vertices has n − 1 edges.

We did special cases of this earlier in the term. Now see it’s always true.

Proof. Since any tree can be rooted and ordered, enough to show this for any rooted, ordered tree. Do this by strong induction on the height.

BASE: height = 0: One node, no edges, true.

INDUCTIVE: Assume true for heights < h, show for height h. Consider arbitrary tree T, height h. Has subtrees T_{1}, …, T_{k}, each of height < h. By strong inductive hypothesis, each has exactly one more vertex than edges.

So subtrees have, in all, k more vertices than edges. Now adding k edges but only one vertex. So end up with 1 more vertex than edges.

Proves inductive step, hence theorem.

Obvious result: If n is number of nodes, i number of internal nodes and l number of leaves, then n = i + l.

What can we say about number of vertices, edges, internal vertices, leaves, etc., in m-ary tree?

First consider special case: full trees. Define full m-ary tree – each internal node has m children. (However, needn’t have all leaves at same level – draw examples.

Theorem 2.2 Suppose that a full m-ary tree (m ≥ 2) has l leaves, i internal nodes, e edges, n total nodes. Then:

1. (A basic relationship for full m-ary trees, n in terms of i.) n = mi + 1.

2. (First consequence, i in terms of n.) i = \frac{n−1}{m}.

3. (l in terms of n) l = \frac{(m−1)n+1}{m}.

4. (n in terms of l) n = \frac{ml−1}{m−1}.

5. (l in terms of i) l = (m − 1)i + 1.

6. (i in terms of l) i = \frac{l−1}{m−1}.

Proof.

1. Every vertex except root is child of an internal vertex.

So n is equal to 1 plus the number of children vertices.

Claim there are exactly mi children vertices, since there are i internal vertices and each has exactly m children.

2. Just solve the previous for i.

3. Combine obvious fact l = n − i and previous part.

4. Solve previous for n.

5. Combine obvious fact and part 1, solve for l.

6. Solve previous for i.

Can say more for the further restricted case: complete trees. And complete m-ary tree (full and all leaves at same level).

Theorem 2.3 Suppose that a complete m-ary tree (m ≥ 2) has h height, l leaves, i internal nodes, e edges, n total nodes. Then:

1. l = m^{h}.

2. i = \frac{m^{h}−1}{m−1}

3. n = \frac{m^{h+1}−1}{m−1}

4. e = \frac{m^{h+1}−1}{m−1} − 1

Proof.

1. l = m^{h}

By induction on h.

BASE: h = 0: 1 = m^{0} leaves.

INDUCTIVE: Assume for h, prove for h + 1.

Given tree T of height h + 1, children subtrees have height h, so by inductive hypothesis, each has m^{h} leaves. Leaves of whole tree are exactly the leaves of the subtrees. So m × m^{h}, which = m^{h+1}, as needed.

2. Have i = \frac{l−1}{m−1}, substitute previous part.

3. Have n = \frac{ml−1}{m−1}, substitute expression for l.

4. By subtracting 1 from the previous.

The preceding theorem gives l, i, n, e in terms of h and m.

Can use these counting results to get upper bounds for non-complete, in fact, non-full m-ary trees, e.g.:

1. n ≤ mi + 1.

2. i ≥ \frac{n−1}{m}.

3. l ≤ \frac{(m−1)n+1}{m}.

4. n ≥ \frac{ml−1}{m−1}.

5. l ≤ (m − 1)i + 1.

6. i ≥ \frac{l−1}{m−1}.

7. l ≤ m^{h}.

8. i ≤ \frac{m^{h}−1}{m−1}

9. n ≤ \frac{m^{h+1}−1}{m−1}

10. e ≤ \frac{m^{h+1}−1}{m−1} − 1

LTTR to prove these.

2.3 Applications of Trees

2.3.1 Binary Search Trees

To aid in storing and searching for data.

Store data with keys from totally ordered set, at nodes of ordered binary tree. Do this to make it fast to insert and look up items using their keys.

Invariant property to maintain: Any value occurring in left subtree of node is < value at node. Any value in right subtree of node is > value at node.

Example: Fig. 6

Fast for searching. E.g. to see if t is in tree, compare from top, go left if less, go right if greater, etc. Trace and reach it. E.g., to see if n is in tree, trace – not there. The beauty is that the whole tree need not be searched, just one path. Usually faster, unless tree is very skewed.

Fast also for insertion of new data. Trace.

Deletion may leave empty nodes, needs special reorganization methods. See in algorithms class 6.046.

2.3.2 Compiling

Very important use of trees is to represent programs, and pieces of programs, internally during the compilation process. Trees give the structure of the program, which is not made explicit in a linear text representation.

Example: Arithmetic expressions

E.g., (2 + x) - (3 - (x + y)) written as ...Fig. 7 Ordered tree.

Example: More interesting program structure:

if x > y then while x <= 5 do x := x+1; a:= a^2

Draw, Fig. 8

This kind of tree is called a parse tree. It is produced during the compilation process, by a compiler program.

Besides containing the given information, it gets decorated with other info, like the types of the variables. It is then used as the basis for generating machine code.

2.3.3 Decision Trees

Can represent the decision structure of a program. Unwind loops, represent as tree.

Example: Sorting a list of 4 numbers a, b, c, d (assume unequal): Fig 9:

Answers at bottom. Height of tree corresponds closely to worst-case running time.

2.4 Spanning Trees

[[[Skip this? What could be done here: Definition of spanning tree. Existence of spanning tree for arbitrary graph, simple method for finding it, by searching.

Then define weighted spanning tree. Give the basic theorem about being able to add to any forest that’s a part of a MST to get another MST. Give two example algorithms, Prim and Kruskal. Could explore this in tutorial.]]]

3 State Machines

Now start the final discrete structure we will study in this course: “state machines”. Continue next time.

Used as the basis for modelling algorithms and physical machines, in order to give precise specifications of their behavior and in order to verify their properties.

Will present: Definitions of state machines, executions, reachability.

Basic examples: Illustrate how to use state machines to represent interesting computational processes. Decomposition of state into variables, for convenience in representation and manipulation. Deterministic vs. nondeterministic state machines.

Then introduce the key notion of “invariants”: Properties that are always preserved by state machines. These are the main tool used in practice, in careful reasoning about algorithms and machines.

Show how invariants can be proved. By induction. Some that don’t need strengthening of inductive hypothesis. Others that do. Proofs of invariants in batches.

Examples: Numbers: hopscotch Euclid’s GCD algorithm fast multiplication

Posets:

Lecture ??: 12 Bubble sort, or some other sort Graphs: Acyclic graph maintenance Breadth-first search, relaxation Distributed algorithms: Leader election Alternating Bit Protocol

If time, say something about: Safety vs. liveness Progress functions. Temporal logic – box and diamond. Simulation relation

3.1 Definitions: State Machine, Executions and Reachability

A state machine is a triple (Q, Q_{0}, δ), where:

1. Q is a nonempty (finite or infinite) set of states.

2. Q_{0} ⊆ Q is a nonempty subset of the states known as the start states.

3. δ ⊆ Q × Q is a binary relation over Q. Don’t confuse with 6.045 FSM’s – these machines need not be finite. Similar to digraphs, but has distinguished start states.

The point of a state machine is that it executes, producing executions. Starts from any start state, and moves along, following transitions. In this way, it models a computational process. An execution fragment of a state machine is a finite or infinite sequence q_{0}, q_{1}, … such that (q_{i}, q_{i+1}) ∈ δ for every i ≥ 0. An execution is an execution fragment in which the initial state q_{0} is an element of Q_{0}.

A state of Q is said to be reachable if it is the final state of some finite execution.

Simple examples. Begin with a simple state machine with one initial state, no choices of what to do next. Has infinitely many states.

Example 1 Counter

Q = ℕ, Q_{0} = {0}

δ = {(n, n + 1)|n ∈ ℕ}

Intuitively, this models a counting process that starts with value 0 and increments (forever).

The reachable states are all of Q = ℕ. Each natural number n is only reached by an execution with n transitions (n + 1 states). There is only one execution of each number of transitions.

The counter is an example of a special kind of state machine that we call deterministic. This means that it has only a single start state: |Q_{0}| = 1, and each state has at most one successor: for each q ∈ Q, there is at most one q′ such that (q, q′) ∈ δ.

If a state machine isn’t deterministic, we say it’s nondeterminisitic. Means it has choices, either of starting state, transitions, or both.

The following is an example of a nondeterministic state machine:

Example 2 Linear hopscotch, with steps of 12 and 18:

Q = integers Q_{0} = {0}

δ = {(i, i + 12)|i ∈ ℤ} ∪ {(i, i − 12)|...} ∪ {(i, i + 18)|...} ∪ {(i, i − 18)|...}. Thas has many different executions, all leading to states that are multiples of the GCD, which is 6.

Can also vary this machine to allow multiple start states.

E.g., Q_{0} = {i|i = 0 (mod 6)}. E.g., Q_{0} = {0, 2}.

In most examples of state machines that occur in practice, the state has structure. Can describe it as containing different sorts of information, represented as values of state variables. Different from graphs, which are usually thought of as unstructured. (Normally represented by a dot for each vertex, arrow for each edge.) The variable structure leads to concise, understandable representations.

Example 3 2-dimensional hopscotch, say with 12 and 18 in EW and 9 and 12 in NS direction. Q: Consists of values for two variables: east-west ∈ ℤ, initially 0 north-south ∈ ℤ, initially 0

Formally, Q is the Cartesian product, ℤ × ℤ. And Q_{0} consists of the single pair, (0, 0).

δ: All transitions of the following forms:

move-east(12) Effect: east-west := east-west + 12

move-east(18) Effect: east-west := east-west + 18

move-west(12) Effect: east-west := east-west − 12

move-west(18) Effect: east-west := east-west − 18

move-north(9) Effect: east-west := east-west + 9

Etc. The names we associate with the transitions are called labels. They have no formal significance here, but are convenient for presenting the state machine. Formally, δ consists of all pairs that are described by any of the little code fragments. A code fragment describes a set of pairs of states, in an obvious way. E.g., the first fragment denotes all pairs (q, q′), where east-west in q′ is 12 more than it is in q, and north-south is the same in q′ as it is in q.

Written concisely: q′.east-west = q.east-west + 12, q′.north-south = q.north-south.

The transitions above are described using assignment statements. Nice for programmers. Sometimes use equations (like the ones just above) instead.

One more wrinkle: When using code fragments to represent state transitions, sometimes it’s convenient to give a predicate restricting when the code fragment is allowed to be applied:

Example 4 2-dim hopscotch on only northeast quadrant:

The same as above, only now the two state components east-west and north-south take on nonnegative values only (values in ℕ).

So now we have to be careful – some of the named transitions can’t be applied in all situations, e.g.:

move-west(12) Precondition: east-west ≥ 12 Effect: east-west := east-west − 12

The precondition is a predicate on the state variables restricting when the transitions denoted by this code fragment are permitted to occur. Formally, the transitions denoted by this fragment are all (q, q′) for which the precondition predicate is true of q and for which q′ is obtained from q by executing the effect code.

State machines can be used to describe most computing processes, including both algorithms and computing machines. The state variable structure is usually crucial for obtaining understandable respresentations.

3.2 Invariants

Now we are ready to introduce the main practical method for reasoning about the correctness of algorithms/machines represented as state machines.

Argue that the machine preserves some invariants. This is often the key to understanding what the machine does, and why.

Definition 3.1 An invariant of a state machine M is any predicate that is true of all the reachable states of M.

Example 5 In the general 2-dim hopscotch, with initial state (0, 0), the statements that east-west is a multiple of 6, and north-south is a multiple of 3, are invariants.

Of course, these statements give a lot of insight into what this machine does.

How to prove that some state machine satisfies an invariant? Must be true of all reachable states. Reachable states are those that are reached by some number of transitions, beginning from a start state. So, prove by induction on number of transitions. Breaks down into base, 0 transitions, which shows the property is true in the initial states, and inductive step, which shows the property is preserved by all steps.

Example 6 2-dimensional hopscotch.

Invariant as stated above. Prove by induction

BASE: Initial state (0, 0) multiples of 6, 3 respectively. INDUCTIVE STEP:

Suppose true for all states reachable after n steps. Now consider state reachable after n + 1 steps. That state is reached by an execution with n steps, then one more step. Say the state after n steps is q and the final state is q′. By inductive hypothesis, we may assume that the invariants are true for q. That is, q.east-west is a multiple of 6 and q.north-south is a multiple of 3. We must show that q′.east-west is a multiple of 6 and q′.north-south is a multiple of 3.

Consider cases based on type of transition. (The usual, when the machine is represented by code fragments.)

Here, all the cases are basically the same, so I’ll just do the first. Since q.east-west is a multiple of 6 and q′.east-west = q.east-west + 12, we also have q′.east-west a multiple of 6. Also (don’t forget) since q.north-south is a multiple of 3 and q′.north-south = q.north-south, we also have q′.north-south a multiple of 3.

QED

Next time, we’ll do more examples. First, number theory. Then posets and graphs. Then some arising from distributed systems.