\documentclass[11pt,twoside]{article}
\usepackage{macros-course-lectures}

%%% Already defined
%\newtheorem{falsethm}[theorem]{False Theorem}
\renewcommand{\thepage}{\arabic{page}}

%\makeslides

\begin{document}

\onlystudents

\lecture{notes6}{Relations, Equivalence Relations, and Partial Orders}{February 17, 2000}

\onlylecture{{\bf was lecture too slow last time?}}

\section{Relations}

Last time we talked about
\begin{itemize}
\item relations
\item properties: they can be reflexive, symmetric, antisymmetric,
  transitive
\item  3 ways of representing them: lists of tuples, matrices, and graphs
\end{itemize}

Something I forgot to mention last time: when we talk about we
relations as graphs, we change terminology: elements of $A$ are {\em
  vertices} and pairs in the relation are \emph{edges}.  (This is
inherited from geometry, where graphs also play an important role.)

\subsection{Operations on Relations}

Notice the difference between properties of relations 
and operations on relations\. A property, such as 
symmetry or transitivity, is something a relation 
may or may not satisfy\. An operation on relation, 
such as relation composition we are going to define, 
is a way to combine one or more relations to build 
some other relation.

\subsubsection{Set Operations}

Relations are sets of tuples, so we can apply set operations\.
The intersection of two relations gives pairs satisfying both relations\.
The union gives pairs satisfying one---the union of ``brother-of'' and
``sister-of'' is ``sibling-of''\.
Complement gives pairs not satisfying\.

\subsubsection{Inverse}

If $R$ is a relation on $A \times B$, then $R^{-1}$ is a relation on
$B\times A$ given by $R^{-1} = \set{(b,a) \mid (a,b) \in R}$.  It's
just the relation ``turned backwards.''

Inverse of ``parent-of'' is ``child-of.''


\subsubsection{Composition}


The {\em composition} of relations $R_1 \subseteq A \times B$ and $R_2
\subseteq B \times C$ is the relation $R_2 \circ R_1 =\{(a,c) \mid
(\exists b) ((a,b) \in R_1) \wedge ((b,c) \in R_2)\}$. \\
Notice that $R_1\circ R_2$ and $R_2\circ R_1$ are different.
This is the notation used in the text book. 
Some people prefer to write $R_1\circ R_2$ for 
the composition of $R_1$ and $R_2$, which may seem 
more natural,
but from now on we will stick with the book.

Examples: \\
Composition of parent-of relation with itself gives grandparent-of. \\
Composition of child-of relation with parent-of relation gives
sibling-of relation. \\
Does composition of parent-of with child-of give married-to/domestic
partners?  No, because misses childless couples.
\\
\\
Composition of relations is equivalent to matrix multiplication, with
$+$ replaced by $\vee$ (Boolean or) and $*$ replaced by $\wedge$. \\
\\
Ex: \\
Let $R_1$ be relation from first example above: \\
\[\begin{array}{r|ccc}
& a&  b&  c\\
\hline  
0& 1&  0&  1\\
1& 1&  0&  1\\
2& 0&  1&  0\\
3& 0&  0&  0\\
\end{array}
\]

Let $R_2$ be relation from $\{ a, b, c \}$ to $\{ d,e,f \}$ given by:
\[\begin{array}{r|ccc}
& d&  e&  f\\
\hline  
a& 1&  1&  1\\
b& 0&  1&  0\\
c& 0&  0&  1\\
\end{array}
\]

Then $R_2 \circ R_1 = \{ (0,d), ... \}$, 
that is, \\
\[\begin{array}{r|ccc}
& d&  e&  f\\
\hline  
0& 1&  1&  1\\
1& 1&  1&  1\\
2& 0&  1&  0\\
3& 0&  0&  0\\
\end{array}
\]

Same as matrix multiply, using Boolean ops and and or. \\
\\
\\
Relational composition is associative. \\
Just like matrix multiplication. \\

\begin{lemma}
  \label{associative}
  $R_1 \circ (R_2 \circ R_3) = (R_1 \circ R_2) \circ R_3$
\end{lemma}

The composition $R \circ R$ of $R$ with itself is written $R^2$. \\
Similarly $R^n$ denotes $R$ composed with itself $n$ times. \\
Recursive definition: \\
$R_1 = R$, $R^n = R \circ R^{n-1}$. \\
Actually, can define $R_0 = \{ (a,a) \mid a \in R \}$ as base case. \\
We'll just use $1$ as base case, as book does. \\
\\
Could just as well have defined as $R^{n-1} \circ R$:

\subsubsection{Closure}

A closure ``extends'' a relation to satisfy some property.  But
extends it as little as possible.

\begin{definition}
  The \emph{closure} of relation $R$ with respect to property $P$ is
  the relation $S$ that (i) contains $R$, (ii) has property $P$, and
  (iii) is contained in \emph{any} relation satisfying (i) and (ii).
  That is, $S$ is the ``smallest'' relation satisfying (i) and (ii).
\end{definition}

There might be no one relation satisfying the definition; in that case
the closure is not defined.

\begin{lemma}
  The reflexive closure of $R$ is $S=R \cup \set{(a,a) \mid a \in A}$\ 
\end{lemma}
\begin{proof}
  It contains $R$ and is reflexive by design\.  Furthermore (by
  definition) any relation satisfying (i) must contain $R$, and any
  satisfying (ii) must contain the pairs $(a,a)$, so any relation
  satisfying both (i) and (ii) must contain $S$.
\end{proof}

\begin{lemma}
  The symmetric closure of $R$ is $S=R \union R^{-1}$.  
\end{lemma}
\begin{proof}
  This relation is symmetric and contains $R$\.  It is also the
  smallest such\.  For suppose we have some symmetric relation $T$
  with $R \subseteq T$\.  Consider $(a,b) \in R$\. Then $(a,b) \in T$
  so by symmetry $(b,a) \in T$\.  It follows that $R^{-1} \subseteq
  T$\.  So $S=R \cup R^{-1} \subseteq T$.
\end{proof}

For the transitive closure, we need to introduce some new terminology.

\begin{definition}
  A \emph{walk} in a relation $R$ is a sequence $a_0,\ldots,a_k$ with
  $k \ge 1$ such that $(a_i,a_{i+1}) \in R$ for every $i < k$.  We
  call $k$ the \emph{length} of the walk.
\end{definition}

In the graph model, a walk is something you can trace out by following
arrows from vertex to vertex, without lifting your pen\.  Note that a
singleton vertex is \emph{not} a length 0 walk (this is just for
convenience).

Some books call the above a \emph{path}.  But this is ambiguous
because of the following definition:
\begin{definition}
  A \emph{simple path} is a walk with no repeated vertices.
\end{definition}
Many people say path when they mean simple path, so we will use walk
to allow for possibly non-simple paths.

Walks give us the terminology for transitive closure.

\begin{lemma}
  The transitive closure of a relation $R$ is that set 
  \[
  S=\set{(a,b) \mid \mbox{there is a walk from $a$ to $b$ in $R$}}.
  \]
\end{lemma}
\begin{proof}
  First let's show $S$ is transitive\.  Suppose $(a,b)\in S$ and
  $(c,d) \in S$\.  This means that there is an $(a,b)$ walk and a
  $(b,c)$ walk in $R$\.  If we ``concatenate'' them (attach the end of
  the $(a,b)$ walk to the start of the $(b,c)$ walk, we get an $(a,c)$
  walk\.  So $(a,c) \in S$\.  So $S$ is transitive.

  So consider any transitive relation $T$ containing $R$\.  We have to
  show it contains $S$\.  Assume for contradiction that $S \not
  \subseteq T$\.  This means that some pair $(a,b) \in S$ but $(a,b)
  \notin T$\.  In other words, there is a walk $a_0,\ldots,a_k=b$ in
  $S$ where $(a_0,a_k) \notin T$\.  Call this a ``missing'' walk.

  Now we use well ordering (i.e. induction)\.  We have just claimed that the
  set of walks in $T$ is nonempty\.  So consider a shortest missing walk
  $s_0,\ldots,s_m$\.  We will derive a contradiction to this being the
  shortest missing walk\.  

  First suppose $m=1$.  The $s_0,s_1$ is a walk in $R$, so $(s_0,s_1)
  \in R$\.  But we know $T$ contains $R$, so $(s_0,s_1)\in T$, a
  contradiction.

  Now suppose $m>1$\.  Then $s_1,\ldots,s_{m-1}$ is a walk in $R$ ($m-1
  > 0$)\.  But it is shorter than our original ``shortest missing''
  walk, so cannot be missing\.  Thus $(s_1,s_{m-1}) \in T$\.  Also we
  have $(s_{m-1},s_m) \in T$ since $R \subseteq T$\.  Thus by
  transitivity of $T$, $(s_1,s_m)\in T$, a contradiction.

  We get a contradiction either way, so our assumption (that $S
  \not\subseteq T$) is false.  This proves the theorem.
\end{proof}

Wait a minute.  Well ordering is applied to sets of \emph{numbers}; we
applied it to a set of walk!  How\?  Well, look at the set of
``lengths of missing paths''\.  It is nonempty, so has a smallest
element\.  There is walk that has this length---so it is a shortest
walk.


\subsection{Computing the Transitive Closure}

With reflexive and symmetric closure, it is pretty clear how to
actually build them\.  But for transitive closure, how do we actually
find all the walks we need?

Let's start by finding walks of a given length\.  Recall the definition of
$R^n$ of $R$ composed with itself $n$ times.

\begin{lemma}
$R^n = \set{(a,b) \mid \mbox{ there is a length $n$ walk from $a$ to
    $b$ in $R$}}$
\end{lemma}
\begin{proof}
  By induction\.  The base case is clear---walks of length one are
  edges of $R$\.  Suppose it is true for $R^n$; let's prove it for
  $R^{n+1}$\.  Suppose there is a walk $a_0,\ldots,a_{n+1}$ in $R$\.
  This is a walk $a_0,\ldots,a_n$ in $R$ followed by a tuple
  $(a_n,a_{n+1})$ of $R$\.  Thus (by induction) we have $(a_0,a_n) \in
  R^n$ and $(a_n,a_{n+1})\in R$\.  Thus (by definition of composition)
  $(a_0,a_{n+1}) \in R^{n+1}$ as claimed.

  Am I done?  No, I only proved set containment in one direction\.  I
  still need to show that if  $(a,b) \in R^{n}$ there is a
  walk of length $n$ from $a$ to $b$\.  But I'll let you do it: use
  the induction above, but turn the proof backwards.
\end{proof}

This means we can write the transitive closure of $R$ as $R \cup R^2
\cup R^3 \cdots$\.  Better (since we know how to do composition), but
still a problem: there are infinitely many terms!

\begin{lemma}
Suppose $A$ has $n$ elements.  If there is a walk from $A$ to $B$,
there is a walk of length at most $n$ from $A$ to $B$.
\end{lemma}
\begin{proof}
  We'll use well ordering (for a change)\.  Consider the shortest walk
  $a=a_0,a_1,\ldots,a_k$ from $a$ to $b$ (we know one exists, so by
  well ordering there is a shortest one)\.  Suppose $k > n$\.  Then
  some element of $A$ appears twice in the list (with more than $n$
  list entries, one must be a repeat)\.  This means the walk is at
  some point circling back to where it was before\.  We can cut out
  this cycle from the walk and get a shorter walk\.  This contradicts
  that we had a shortest walk.  So we cannot have $k>n$.
\end{proof}

So we don't need infinitely many terms.  It is enough to take walks of
length $n$, namely $R^1\cup R^2\cdots\cup R^n$.


\section{Equivalence relations and partitions}

Now we consider an important special type of relation, called an {\em
  equivalence relation}\.  We give an important special case --
modular arithmetic\.  We show an application to RSA.

\subsection{Definitions}

\begin{definition}
  An {\em equivalence relation} is a relation that is reflexive,
  symmetric and transitive\.
\end{definition}

For example, the ``roommates'' relation is an equivalence relation.
So is ``married to'', ``same size as'', and ``on same Ethernet hub''\.
A trivial example is ``='' relation on natural numbers\. 
The hallmark of equivalence relations is the word ``same''\.  It
provides a way to ``hide'' unimportant differences.

Equivalence relations partition the universe into subsets in a natural
way:\\
\begin{definition}
  A {\em partition} of a set $A$ is a collection of subsets
  $\{A_1,\cdots,A_k\}$ such that any two of them are disjoint (for any
  $i\neq j$, $A_i\cap A_j=\emptyset$) and such
  that their union is $A$. \\
\end{definition}
That is, every element of $A$ is in exactly one subset. \\
\\
Fix an equivalence relation $R$ on $A$. \\
For an element $a\in A$, let $[a]$ denote the set $\{b\in A \given
a\sim_R b\}$. \\
We call this set the {\em equivalence class of $a$ under $R$}\.
We call $a$ a {\em representative} of $[a]$\\

\begin{lemma}
  The sets $[a]$ for $a\in A$ constitute a partition of $A$.  That is,
  for every $a,b\in A$, either $[a]=[b]$ or $[a]\cap [b]=\emptyset$.
\end{lemma}

\begin{proof}
  Fix $a,b\in A$. \\
  If $[a]=[b]$ then we are done, so suppose not. \\
  Let $c$ be any element that is in one of the sets but not the other. \\
  Without loss of generality we can assume that $c \in [b]-[a]$
  (we know that either $c\in [b]-[a]$ or $c\in [a]-[b]$.
  In the latter case we can simply swap $a$ and $b$ and reduce to 
  the first case.).
  \\
  Similarly, if $[a] \cap [b]=\emptyset$ then we are done, so suppose
  not. \\
  Let $d$ be any element in $d \in [a] \cap [b]$. \\
  \\
  We will get a contradiction by showing that $a\sim_R c$ and therefore
  that $c\in [a]$. \\
  \\
  First, $a\sim_R d$ because $d\in [a]$ (note that $d=a$ is a possibility but
  this is ok because $R$ is reflexive). \\
  Second, $d\sim_R b$ and $b\sim_R c$ because both $c,d\in [b]$ and $R$
  is symmetric. \\
  This implies, by transitivity, that $d\sim_Rc$. \\
  Finally, by transitivity, $a\sim_R c$ because $a\sim_R d$ and $d\sim_R c$.
\end{proof}

Conversely, any partition $\{A_1,\cdots,A_k\}$ of $A$ defines an
equivalence relation by letting $a\sim_R b$ iff $a$ and $b$ are in the
same $A_i$. \\
(Note ``same''.) \\
This is an equivalence relation:\\
\begin{itemize}
\item Reflexivity: for all $a$ we know $a\in A_i$ for some $i$,
  by definition of partition. Clearly $a$ and $a$ are in the same $A_i$,
  and $a\sim_R b$.
\item
  Symmetry: Assume $a\sim_R b$, that is $a$ and $b$ are in the 
  same $A_i$. Also $b$ and $a$ are in the same $A_i$
  and therefore $b\sim_R a$.
\item
  Transitivity: Assume $a\sim_R b$ and $b\sim_R c$.
  By definition of $\sim_R$, $a$ and $b$ are in the same 
  $A_i$ for some $i$, and $b$ and $c$ are in the same $A_j$
  for some $j$.
  But by definition of partition $b$ cannot be in two different $A_i$'s.
  So, it must be $A_i=A_j$ and $a$ and $c$ are in the same 
  $A_i$, proving $a\sim_R c$.
\end{itemize}

Therefore, we can look at partitions and at equivalence relations as
the same thing. 

\subsection{Integers modulo $m$}

An equivalence relation on integers (positive, negative and $0$):

\begin{defin}
  If $a$ and $b$ are integers, then we say $a=b \pmod m$ if $m \mid
  (a-b)$.  
\end{defin}

An equivalent formulation says that $a=b+km$ for some integer $k$.  

\begin{theorem}
  Equality modulo $m$ is an equivalence relation.
\end{theorem}

\begin{proof}
  We need to show the relation is reflexive, symmetric, and transitive.
  \begin{itemize}
  \item {Reflexive:} Clearly $m \mid a-a = 0$, so $a=a \pmod m$.
  \item {Symmetric:} If $a=b+km$ then $b=a+(-k)m$.
  \item {Transitive:} Suppose $a=b \pmod m$ and $b = c \pmod m$\. 
    Then $m \mid (a-b)$ and $m \mid (b-c)$\.  So $(a-b)=k_1 m$ and
    $(b-c)=k_2 m$\.  So $(a-c)=(a-b)+(b-c)=(k_1+k_2)m$\.  Therefore $m
    \mid = a-c$.
  \end{itemize}
\end{proof}

The equivalence class of $a$ is simply the set 
$[a] = \{ b \in {\Bbb Z} \mid a = b \pmod m \}$, or
$\{ km + a \mid k \in {\Bbb Z}\}$. \\
\\
It turns out that we can extend a lot of standard arithmetic to work
modulo $m$. \\
In fact, we can define notions of sum and product for the equivalence
classes mod $m$. \\
For example, we define $[a]+[b]$, given two equivalence classes mod
$m$, to be the equivalence class $[a+b]$. \\
This is not as obvious as it seems: notice
that the result is given in terms of $a$ and $b$, two selected
representative from the equivalence classes, but that it is supposed
to apply to the equivalence classes themselves.  \\
To prove that this works, we have to show that it doesn't matter which
representatives of the equivalence classes we choose:

\begin{lemma}
  If $a=x \pmod m$ and $b=y \pmod m$ then $(a+b) = (x+y) \pmod m$.  
\end{lemma}

\begin{proof}
  $m \mid (a-x)$ and $m \mid (b-y)$ so $m \mid ((a-x)+(b-y)) =
  (a+b)-(x+y)$,  
\end{proof}

It follows that if we are interested only the result of the addition
modulo $m$, which is the case, for example, in the RSA cryptosystem,
that we can at any time replace a given number with a different number
equivalent to it, without changing the value (equivalence class) of
the final answer. \\
\\
The same fact can be proven for multiplication.  \\
This is the basis of the fast exponentiation algorithm presented in
the next subsection.

\subsection{Fast Exponentiation}

Recall that for RSA we wanted to compute the remainder of $x^a$\. on
division by $b$---that is, the equivalence class $\bmod b$ of $x^a$.


In the section on recursive algorithms, we already saw how to reduce
the number of multiplications substantially\.  But there is still a
problem: $x^a$ is a really big number\.  It is bigger than $2^a$, so
has $2^{2^128}$ bits if we use a 128-bit key $a$\.  It will take a
long time to write this number down (if we could find a place to fit
it)\.  

But notice that we only care about the remainder modulo $b$.  So using
modular arithmetic we can solve the large-number problem\.  Whenever
the number we are computing gets too big, replace it with a smaller
representative of the same equivalence class\.  What is the smallest
representative?  The remainder on division by $b$\.  We've argued that
switching representatives doesn't change the answer\.  But now all our
math is done on number smaller than $b$---i.e. 128-bit numbers.

\section{Partial Orders}

A particular type of binary relation. \\
\\
Important in computer science. \\
Applications to task scheduling, \\
Database concurrency control, \\
Logical time in distributed computing \\

\subsection{Definitions}

\subsubsection{Partial Order Definition}

\begin{defin} 
  A binary relation $R \subseteq A \times A$ is a {\bf partial order\/}
  if it is reflexive, transitive, and antisymmetric.
\end{defin}

Quick review:
\begin{itemize}
\item reflexive: $a R a$
\item transitive: $a R b \wedge b R c \Rightarrow a R c$
\item symmetric: $a R b \Rightarrow b R a$
\item antisymmetric: $a R b \wedge b R a \Rightarrow a=b$.
  i.e., $\forall a \neq b, a R b \Rightarrow \neg b R a$. Means NEVER
  symmetric!
\end{itemize}

For comparison: \\
Recall that a relation that is reflexive, transitive, and symmetric
is an equivalence relation. \\
\\


The reflexivity, antisymmetry and transitivity properties are abstract
properties that generally describe ``ordering'' relationships. \\ 

So we often write an ordering-style symbol like $\preceq$ instead of
just a letter like $R$, for a p.o. relation.  This lets us use
notation similar to for $\le$.  For example, we write $a \prec b$ if
$a \preceq b$ and $a \ne b$.  Similarly, we write $b \succeq a$ as
equivalent to $a \preceq b$.

But this could be misleading -- note that $\geq$ is a p.o. on natural
numbers, as well as $\leq$.  If we use the $\preceq$ symbol for $\geq$,
things look really funny. 
In cases like this, better to use $R$.
\\
A partial order is always defined on some set $A$.  The set together
with the partial order is called a ``poset'':
\begin{defin}
  A set $A$ together with a partial order $\preceq$ is called a 
  {\bf poset\/}  $(A, \preceq)$.
\end{defin}



Examples of posets:
\begin{itemize}
\item $A=\naturals, R=\leq$, easy to check reflexive, transitive, antisymmetric
\item $A=\naturals, R=\geq$, same.
\item $A=\naturals, R=<$, NOT because not reflexive
\item $A=\naturals, R=|$ (divides), easy to check reflexive,
  transitive, antisymmetric
\item $A=P(\naturals), R=\subseteq$, check reflexive: $S \subseteq S$,
  transitive: $S \subseteq S' \wedge S' \subseteq S'' \Rightarrow S
  \subseteq S''$, antisymmetric: $S \subseteq S' \wedge S' \subseteq S
  \Rightarrow S = S'$.
\item $A=$ ``set of all computers in the world'', $R=$ ``is (directly or
  indirectly) connected to''. \\
  NOT a p.o. because it is not true that $a R b \wedge b R a \Rightarrow
  a=b$. \\
  In fact, it is symmetric, transitive.  Equivalence relation.
\item $A=$ ``set of all propositions'', $R=\Rightarrow$, NOT because
  not antisymmetric. Not symmetric either, so not equivalence
  relation.
\end{itemize}


\subsubsection{Directed Aciclic Graphs}

A common source of partial orders in computer science is in ``task
graphs''\.  You have a set of tasks $A$, and a relation $R$ in which
$aRb$ means ``$b$ cannot be done until $a$ is finished''\.
Implicitly, ``if all the things that point at $b$ are done, I can do
$b$.''

This can be nicely drawn as a graph\.  We draw an arrow from $a$ to
$b$ if $a R b$\.

For example, below is a graphs that describes the order in which one
would put on clothes\. The set is of clothes, and the edges say what
should be put on before what.

\setlength{\unitlength}{0.012500in}%
                                %
\begingroup\makeatletter\ifx\SetFigFont\undefined
                                % extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
  \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
  \csname \romannumeral\the\count@ pt\expandafter\endcsname
  \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(220,75)(20,760)
  \thinlines
  \put(165,817){\vector( 0,-1){ 20}}
  \put(240,817){\vector( 0,-1){ 20}}
  \put(240,787){\vector( 0,-1){ 20}}
  \put(165,787){\vector( 0,-1){ 20}}
  \put( 25,797){\vector( 0,-1){ 25}}
  \put( 85,797){\vector( 0,-1){ 25}}
  \put(180,787){\vector( 3,-1){ 51}}
  \put(178,765){\vector( 1, 0){ 53}}
  \put(153,795){\vector(-4,-1){88}}
  \put(155,790){\makebox(0.1111,0.7778){\SetFigFont{5}{6}{rm}.}}
  \put(153,790){\vector(-2,-1){ 32}}
  \put( 20,800){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}left sock}}}
  \put( 20,765){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}left shoe}}}
  \put( 80,800){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}right sock}}}
  \put(235,820){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}shirt}}}
  \put(155,820){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}underwear}}}
  \put(235,790){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}sweater}}}
  \put(235,760){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}jacket}}}
  \put(155,760){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}belt}}}
  \put(155,790){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}pants}}}
  \put( 80,765){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}right shoe}}}
\end{picture}

The ``depends on'' graph imposes an ordering on tasks.  But is it a
partial order\?  No, because it isn't reflexive or transitive\.  But
there is a natural extension: the reflexive transitive closure
\emph{is} a partial order\.  It gives the relation ``must be done
before.''

Wait, we know the reflexive transitive closure is reflexive and
transitive, but is it antisymmetric\?
What if I add a relation edge from belt to underwear\?  In that case my
dependency graph stops making sense: there is no way to get dressed!

What goes wrong?  A cyclic dependency.

\begin{definition}
A \emph{cycle} is a walk that ends where it started (i.e., the last
vertex equals the first).
\end{definition}

\begin{definition}
A \emph{directed acyclic graph (DAG)} is a directed graph with no cycles.
\end{definition}

\begin{lemma}
Any partial order is a DAG.
\end{lemma}
\begin{proof}
  Suppose a partial order $R$ has a cycle $a_1,\ldots,a_k,a_1$\.  Then
  by transitivity of $R$ (with an induction hiding inside) we have
  $(a_1 R a_k)$.  We also have $a_k R a_1$.  This violates the
  antisymmetry of $R$, a contradiction.
\end{proof}

\begin{lemma}
The transitive reflexive closure of a DAG is a partial order.
\end{lemma}
\begin{proof}
  Let the DAG be $R$ and its transitive closure $S$\.  We already
  proved that $S$ is the pairs $(a,b)$ that are connected by a walk in
  $R$\.  $S$ is transitive; we just need to prove it is
  antisymmetric\.  We do so by contradiction.  Suppose $A$ is not
  antisymmetric.  Then for some $a$ and $b$, we have $aSb$ and $bSa$\.
  In other words, there is a walk from $a$ to $b$ and a walk from $b$
  to $a$\.  If we attach these two walks, we get a walk from $a$ to
  $a$\.  This contradicts the assumption that $R$ is acyclic.
\end{proof}


\subsubsection{Hasse Diagrams}

If we draw the graph of a poset, we may get \emph{lots} of edges
(consider the $\le$ order on some natural numbers)\.  But a lot of
those edges are ``implicit'' from the transitivity of the order\.  
We can get a much less messy picture by leaving out these arrows.

Consider the clothing example\.  Under the transitive closure,
underwear precedes belt\.  But we don't need to see that edge to know
it is there\.
Similarly loops (which express reflexive relationships) aren't shown.

\begin{defin}
  A {\bf Hasse diagram\/} for a poset $(A,\preceq)$ is a graph: \\
  - whose vertices are the elements of $A$, \\
  - whose edges (arrows) represent pairs $(a,b)$, where $a \leq b$ but
  $a \neq b$, \\
  - that contains no ``transitive edges'', \\
  - for which all pairs in $\preceq$ are obtainable by transitivity from
  edges in the diagram.
\end{defin}

It's a ``transitive disclosure'' of the poset.

To convert the diagram above to an ``official form'' Hasse diagram,
we must remove the pants $\rightarrow$ jacket arrow. 


\subsubsection{Partial vs. Total Orders}

A partial order is called {\em partial \/} because it is not necessary
that an ordering exist between every pair of elements in the set. \\
\\
E.g., Lshoe and Rshoe have no prescribed ordering between them. \\
\\
E.g., For two sets, it's not necessary that either be a subset of the
other. \\
\\
When there is no prescribed order between two elements we say that they
are ``incomparable''. 

\begin{defin}
  $a$ and $b$ are {\bf incomparable\/} if neither $a \preceq b$ nor $b
  \preceq a$.
\end{defin}

\begin{defin}
  $a$ and $b$ are {\bf comparable\/} if $a \preceq b$ or $b \preceq a$.
\end{defin}

E.g., for subsets of $\naturals$, $\{ 1,2,3 \}$ and $\{ 2,3,4 \}$ are
incomparable\.  However, a partial order need not have incomparable
elements\.  As a special case, we can have a p.o. in which there is a
specified order between every pair of elements\.  Such partial orders
are called ``total orders.''

\begin{defin}
  A poset $(S, \preceq)$ is {\bf totally ordered\/} if $(\forall a,b
  \in S) [ a \preceq b \vee b \preceq a ]$.
\end{defin}

Examples:
\begin{itemize}
\item $S=\naturals, \preceq = \leq$
\item $S=\naturals, \preceq = |$, NOT because neither $3|5$ nor $5|3$
\item $S=P(\naturals), \preceq = \subseteq,$ NOT because neither $\{3\}
  \subseteq \{5\}$ nor $\{5\} \subseteq \{3\}$ 
\item Lexicographic order on pairs of natural numbers, defined by:
  $(a_1,a_2) \preceq (b_1,b_2)$ if and only if either $a_1 < b_1$, 
  or else $a_1 = b_1$ and $a_2 \leq b_2$. \\
  Do some examples here. $(2,37) \preceq (2,38) \preceq (3,2)$. \\
  ``Lexicographic'' because like dictionary.
\end{itemize}

The Hasse diagram of a total order looks like a line. 

This has been the basic material. \\
Book has a bit more: \\
minimal, maximal elements (nothing smaller, nothing larger) \\
lower bounds, upper bounds (for a subset -- $\leq$ everything in the
subset, or $\geq$) \\
lub, glb, \\
lattices. \\
Read these parts. 
\\

\subsection{Topological Sorting}

Sometimes when we have a partial order, e.g., of tasks to be
performed, we want to obtain a consistent total order\.  That is, an
order in which to perform all the tasks, one at a time, so as not to
conflict with the precedence requirements. \\ 
\\
The task of finding an ordering that is consistent with a partial
order is known as ``topological sorting''.
I think because the sort is based only on the shape (topology) of the
poset, and not on the actual values.

\begin{defin}
  A {\bf topological sort\/} of a finite poset $(A, \preceq)$ is a
  total ordering of all the elements of $A$, $a_1, a_2, \cdots, a_n$
  \\ in such a way that $\forall i < j$, either $a_i \prec a_j$ or
  $a_i$ and $a_j$ are incomparable in $(A, \preceq)$ (equivalently, it
  is not the case that $a_j \prec a_i$).
\end{defin}

For example, underwear, shirt, Lsock, Rsock, pants, sweater, Lshoe, Rshoe,
belt, jacket is a topological sort of how to dress. \\
\\
One of the nice facts about posets is that such an ordering always
exists and is even easy to find:

\begin{theorem}
  Every finite poset has a topological sort.
\end{theorem}

We'll prove this by induction. \\
On the number of elements in the domain of the poset. \\
The basic idea is to pick a ``smallest'' element to start and then
proceed recursively.

\begin{defin}
  A {\bf minimal\/} element $a$ in a poset $(A, \preceq)$ is one for
  which $(\forall b \in A) [ a \preceq b \vee a \mbox{ and } b \mbox{ are
    incomparable }]$.  Equivalently, it is an element $a$ for which no $b
  \prec a$.
\end{defin}

Notice that there can be more than one minimal element. There are 4 in
the clothes example: Lsock, Rsock, underwear, shirt.

\begin{lemma}
  Every finite poset $(A,\preceq)$ has a minimal element.
\end{lemma}
\begin{proof}
By induction on the number of elements in the poset\.  The base case
is clear.  Suppose it is true for all size-$n$ posets\.  Consider any
size $n+1$ poset.  Delete some element $a$ from the poset\.  This
leaves a poset on $n$ elements (you can verify this)\.  So the smaller
poset has a minimal element $m$\.  

Now consider two cases.  First suppose $a\not\preceq m$\.  The $m$ is a
minimal element of the whole poset\.  Now suppose $a \preceq m$.  Then
I claim $a$ is a minimal element of the whole poset\.  

To prove this claim we argue by contradiction\.  If $a$ is not minimal
then some $b \preceq a$\.  But then, but transitivity, since $a\preceq
m$, we have $b \preceq m$\.  But this would prevent $m$ from being
minimal in the poset with $a$ removed\.  This contradicts our finding
of $m$.
\end{proof}


Note that an infinite poset might have no minimal element.  Consider
$\integers$.  


Now we can prove the existence of the topological sort.

\begin{proof}
  By (ordinary) induction. \\
  $P(n)$ = any poset with $n$ elements has a topological sort. 
  \\
  Base case $(P(1))$: for a poset with one element the one element makes a
  topological sort. \\
  \\
  Inductive step: Assume $(P(n))$. Consider a poset of $n+1$ elements. \\
  By lemma it has a minimal element $u$. \\
  Consider the same relation on $A-\{u\}$. \\
  It is easy to check that it is still reflexive, transitive, and
  anti-symmetric. \\
  So by the inductive hypothesis, $A-\{u\}$ has a topological sort. \\
  We put $u$ first and now have a topological sort on the whole thing. \\
  (According to the definition of topological sort -- $u$ can't be in
  the wrong order w.r.t. any of the later elements.) \\
  \\
  Thus every finite poset has a topological sort, by induction.
\end{proof}

E.g.: Dressing example: \\
Construct ordering by picking one at a time. \\
In each case, look at the remaining p.o. for remaining elements. \\
Lsock, shirt, sweater, Rsock, underwear, pants, Lshoe, belt, jacket, Rshoe

E.g., subsets of $\{ 1,2,3,4 \}$ \\
$\emptyset$ is the unique minimal element, then have choices, e.g., do: \\
$\{ 1 \}$, $\{ 2 \}$, $\{ 1,2 \}$ \\
$\{ 3 \}$, $\{ 1,3 \}$, $\{ 2,3 \}$, $\{ 1,2,3 \}$, \\
$\{ 4 \}$, $\{ 1,4 \}$, $\{ 2,4 \}$, $\{ 3,4 \}$, \\
$\{ 1,2,4 \}$, $\{ 1,3,4 \}$, $\{ 2,3,4 \}$, $\{ 1,2,3,4 \}$

I'm not sure if infinite posets always have topological sorts---I
think this might be equivalent to the well ordering property, which is
equivalent to the axiom of choice, a hotly debated mathematical axiom.

\subsection{Parallel Task Scheduling}

When elements of a poset are tasks that need to be done and the partial
order is precedence constraints, topological sorting provides us with
a legal way to execute tasks sequentially, i.e., without violating any
precedence constraints. \\
But what if we have the ability to execute more than one task at the
same time? \\
For example, say tasks are programs, partial order indicates data
dependence, and we have a parallel machine with lots of processors
instead of a sequential machine with only one. \\
How should we schedule the tasks? \\
Our goal should be to minimize the total {\em time\/} to complete all
the tasks. \\
(For simplicity, say all the tasks take the same amount of time and
all the processors are identical.)
\\
So: \\
Given a finite poset of tasks, how long does it take to do them all,
in an optimal parallel schedule? \\
Assume that all tasks take 1 unit of time and we have an unlimited number
of processors. \\
\\
Use partial order concepts to analyze this problem. \\
\\
On clothes example, we could do all the minimal elements first (Lsock,
Rsock, underwear, shirt), remove them and repeat. \\
Need lots of hands, or maybe dressing servants. \\
(So do pants and sweater next, and then Lshoe, Rshoe, and belt, and
finally jacket.) \\
\\
Can't do any better, because the sequence shirt, pants, belt, jacket
must be done in that order. \\
A sequence like this is called a chain.

\begin{defin}
  A {\bf chain\/} in a poset is a sequence of elements of the domain,
  each of which is smaller than the next in the partial order 
  ($\preceq$ and $\neq$). 
\end{defin}

Clearly, parallel time $\geq$ length of any chain. \\
This is true, because if we used less time, then by pigeonhole two
tasks in the chain would have to be done at the same time, but by
definition of chain this violates precedence constraints. \\
\\
A longest chain is also known as a {\em critical path}. \\
\\
So we need at least $t$ steps, where $t$ is the length of the longest
chain. \\
Fortunately, it is always possible to use only $t$:

\begin{theorem}
  Given any finite poset $(A, \preceq)$ for which the longest chain
  has length $t$, it is possible to partition $A$ into $t$ subsets,
  $A_1, A_2, \cdots, A_t$ such that 
  $(\forall i \in [1,t]) (\forall a \in A_i) 
  (\forall b \preceq a, b \neq a) 
  [ b \in A_1 \cup A_2 \cup \cdots \cup A_{i-1} ]$.
\end{theorem}

That is, can divide up the tasks into $t$ ``groups'' so that for
each group $i$, all tasks that have to precede tasks in $i$ are in
smaller-numbered groups. 

\begin{corollary}
  For $(A, \preceq)$ and $t$ as above, it is possible to schedule all
  tasks in $t$ steps.
\end{corollary}

\begin{proof}
  $\forall i$, schedule all elements of $A_i$ at time $i$. \\
  This satisfies the precedence requirements, because all tasks that
  must precede a task are scheduled at preceding times. \\
\end{proof}

\begin{corollary}
  parallel time = length of longest chain
\end{corollary}

So it remains to prove the partition theorem:

\begin{proof}
  Use the rule: \\
  Put each $a$ in set $A_i$, where $i$ is the length of the longest chain
  ending at $a$. \\
  \\
  See this works: \\
  It gives just $t$ sets, because the longest path is length $t$. \\
  \\
  Need to show $(\forall i) ( \forall a \in A_i)$ \\
  $(\forall b \preceq a, b \neq q) 
  [ b \in A_1 \cup A_2 \cup \cdots \cup A_{i-1} ]$. \\
  \\
  Proof by contradiction: \\
  If $b \not\in A_1 \cup A_2 \cup \cdots \cup A_{i-1}$ then there is a path
  of length $>i-1$ ending in $b$, \\
  which means there is a path of length $>i$ ending in $a$, \\
  which means $a \not \in A_i$. \\
  This is a contradiction.
\end{proof}

So with an unlimited number of processors, the time to complete all
the tasks is the length of the longest chain.  \\
\\
It turns out that this theorem is good for more than parallel
scheduling. It is usually stated as follows.

\begin{corollary}
  If $t$ is the length of the longest chain in a poset $(A, \preceq)$
  then $A$ can be partitioned into $t$ antichains.
\end{corollary}

\begin{defin}
  An {\bf antichain\/} is a set of incomparable elements.
\end{defin}

This is a corollary because we showed before how to partition the tasks
into sets that could be done at the same time, which means that these
were incomparable. (If any two comparable then one would have to
precede other and so couldn't be done at same time.)

Review:
\begin{itemize}
\item {\bf chain\/}: all elements comparable (total order)
\item {\bf antichain\/}: all elements incomparable
\end{itemize}

\subsection{Dilworth's Theorem}

We can use this to prove a famous result about posets:

\subsubsection{The Basics}

\begin{theorem}
  (Dilworth) $\forall t$, every poset with $n$ elements must have a
  chain of size $ > t$ or an antichain of size $\geq \frac{n}{t}$.
\end{theorem}

Ex: Dressing poset.  $n = 10$.
Try $t = 3$.  Has chain of length $4$. \\
Try $t = 4$.  Has no chain of length $5$, but has antichain of size 
$4 \geq \frac{10}{4}$. \\

\begin{proof}
  Assume there is no chain of length $> t$. \\
  So, longest chain has length $\leq t$. \\
  Then by previous result, the $n$ elements can be partitioned into
  $\leq t$ antichains. \\
  (Can assume exactly $t$, since could have some empty.) \\
  So there is an antichain with $\geq \frac{n}{t}$
  elements. \\
  As needed.
\end{proof}

For instance: 

\begin{corollary}
  Every poset with $n$ elements has a chain of length $> \sqrt{n}$
  or an antichain of size $\geq \sqrt{n}$.
\end{corollary}

Set $t= \sqrt{n}$ and this is immediate.

Posets arise in all sorts of contexts, and when they do Dilworth's
theorem can have interesting implications. 

\subsubsection{Increasing and Decreasing Sequences}

A mathematical example: \\
In any sequence of $n$ different numbers, there is either an
increasing subsequence of length $> \sqrt{n}$ or
a decreasing subsequence of length $\geq \sqrt{n}$. \\
\\
E.g., 6 4 7 9 1 2 5 3 8 has the decreasing sequence 6 4 1 and
the increasing sequence \\ 1 2 3 8.
That is, it has both. \\
\\
We can get this using Dilworth's theorem; the trick is to define the 
appropriate poset. \\
\\
The domain is the set of values in the sequence. \\
\\
For the ordering, define $a \preceq b$ if either $a = b$, or else
($a < b$ and $a$ comes before $b$ in the sequence). \\
\\
Check this is reflexive (stated explicitly), 
transitive, and antisymmetric. \\
\\
A chain corresponds to a sequence that increases in value and moves to
the right, that is, an increasing sequence. \\
\\
But what does an antichain correspond to?

\begin{lemma}
  If $a,b$ are incomparable and $a > b$ then $a$ precedes $b$ in the
  sequence. 
\end{lemma}

\begin{proof}
  By contradiction. \\
  Assume $b$ precedes $a$. \\
  Then $b < a$ and $b$ precedes $a$, so $b \preceq a$ in the partial
  order. \\
  This contradicts our assumption that $a$ and $b$ are incomparable.
\end{proof}

Extend this to more than 2 elements:

\begin{lemma}
  If $a_1, a_2, \cdots , a_t$ are incomparable and $a_1 > a_2 > \cdots
  > a_t$ then $a_1, a_2, \cdots a_t$ form a decreasing subsequence.
\end{lemma}

\begin{proof}
  $(\forall i)$,
  the fact that $a_i > a_{i+1}$ implies that
  $a_i$ precedes $a_{i+1}$ in the sequence, by the previous lemma.
\end{proof}

So given an antichain, arrange the elements so that they are in
decreasing order and the ``left of'' relation follows, giving a
decreasing subsequence.

Dilworth's theorem implies that there is either a chain of size 
$> \sqrt{n}$ or an antichain of size $\geq \sqrt{n}$. \\
By the analysis just above,
this yields either an increasing sequence of length $> \sqrt{n}$
or a decreasing subsequence of length $\geq \sqrt{n}$. 

\subsubsection{Number of Comparisons for Database Setup and Searching}

Consider a ``database'' of records with (distinct) numerical keys. \\
Suppose when records are inserted, some comparisons of these keys
occur, and the database maintains a record of these comparisons. \\
\\
E.g., at one extreme, database could be kept totally sorted. \\
E.g., totally unsorted. \\
\\
Each comparison costs something -- the time to perform it. \\
\\
Now suppose that after this setup, a search is to be performed on the
database to determine if a record with a particular key $k$ appears
anywhere in the database.  \\
The search is supposed to be based on comparisons only. \\
Then the worst-case number of comparisons it takes depends on the
database partial order. \\
\\
E.g., if db is totally sorted, can compare with middle elements, then
middle of halves, etc.  \\
Binary search.  \\
Approximately log n comparisons needed. \\
\\
E.g., if db totally unsorted, then in the worst case, have to compare
with every element.  \\
This is because each comparison with an element in the db gives no
information about whether the sought element might be any of the
others in the db. \\
\\
In fact, if there is an antichain of size $k$, then in the worst case,
the new key could be any of the elements of the antichain. \\
So in the worse case, have to compare with all $k$, since each one
gives no information about the others. \\
\\
See what Dilworth's theorem says about the database partial order. \\
For any $t$, either have a chain of length $> t$, or an antichain of
size $\geq \frac{n}{t}$.  \\
Setting up a chain of length $> t$ requires at least $t$ comparisons. \\
The antichain requires at least $\frac{n}{t}$ comparisons. \\
So this says that we have to pay the cost sometime -- tradeoff --
either we do $> t$ comparisons while setting up the db, or at
least $\frac{n}{t}$ in the worst case to search the db. \\
\\
More refined analysis is possible. \\
That's because reliably setting up a long chain, say length $t$, in
the database actually requires {\em more} than $t$ comparisons.  \\
Because in the worst case, the comparisons won't come out in the most
desirable way, setting up a linear chain. \\
Ex: Compare $a$ and $b$, find out $a > b$.  Then compare $b$ and $c$.
Would be nice if we found out $b > c$.  But we might not -- might get
$c > b$.  Then we don't have a chain of length $3$, but still an
antichain of size $2$.  To get a chain, need an additional comparison. \\
\\
Return to this in analysis of algorithms.

\subsection{Logical Time in Distributed Systems}

One last use of p.o.  \\
No use of powerful theorems here -- just showing an important place
where a p.o. is used as a mathematical model for a real computational
situation.  \\
\\
A distributed computation consists of a sequence of ``steps''
occurring at each of a collection of $n$ ``processes''. \\
In addition to performing local steps, the processes can send messages
to each other. \\
So, have $\ms{send}$ steps and $\ms{receive}$ steps. \\
\\
The processes are mostly working independently. \\
The only interaction is by means of the messages. \\
So, the only dependence among steps (feeding data from one to another,
using results to make decisions, etc.) is by: \\
\begin{enumerate}
\item
  Successive steps of same process.
\item
  Send vs. receive of same message.
\end{enumerate}
Taking reflexive transitive closure leads to partial order describing
the dependence among the computation steps. \\
\\
Basic theorem:  Any topological sort of this p.o. describes a possible
sequence in which all the distributed steps could occur. \\
(No changes in what happens based on reordering incomparable steps.) \\
\\
Example of what one might do with this p.o.: \\
Assign ``logical time'' to every step. \\
Instead of an actual real time. \\
The time is just any assignment of real numbers to the steps that is
consistent with the p.o.:  when one step precedes another in the p.o.,
it must have a smaller logical time. \\
\\
Behaves like real time:  Formally, can reorder the steps in logical
time order, still consistent with the p.o.\\
Then the system is behaving as if the logical time were actually real
time. \\
\\
Can be maintained by simple distributed algorithm. \\
(Piggybacks time on message, adjusts local times in response to what
is heard in some messages.) \\
\\
Although logical time isn't the same as actual real time, for many
purposes, it can be used as an adequate substitute. \\
Logical time can be used, e.g., to timestamp files to keep updated
copies consistent.

\end{document}

\endinput

Problems based on this lecture:

1. (Some basic p.o. defs -- minimal and maximal elements, lower and
upper bounds, etc.)
Section 6.6, Problem 20
and/or Section 6.2, Problem 22

2. (Parallel task scheduling)
Recall the theorem from class that says that if a task graph is 
a partial order with maximum chain length $t$, then the
whole set of tasks can be performed by an unlimited collection of
parallel processors in time $t$.
This theorem was only for the case where all tasks take the same
amount of time, say $1$ time unit.
(a) Apply this theorem to produce an optimal schedule for the
following task graph.
[[[give some graph.  Don't need to make up a cute story -- just some
letters will do]]]
(b) Generalize the theorem to the case where some tasks take $1$ time
unit and others take $2$ times units.
[[[Seems this should work by splitting the time-$2$ tasks into two
tasks, using this to come up with a new length measure for chains,
then applying the old theorem. 
Should take some arguing that everything works, but it seems nice.???]]]

3. This doesn't actually use Dilworth's theorem.
But the students will get to think about chains and antichains.

Suppose a database is stored using a partial order on keys.  
Your job is to design a search strategy to determine if a new key $k$
appears in the database.  
The only thing you are allowed to do in the strategy is to compare two
keys (either two within the database, or the new key $k$ vs. a key in
the database).
The result of such a comparison adds extra edges to the partial order.
The results of earlier comparisons can be used to determine which
comparisons are made later.

For each of the partial orders given below, describe a strategy for
searching for an arbitrary key $k$ in the database.
Your strategy should be designed to use the minimum number you can
achieve, of comparisons, in the worst case.
Say how many comparisons it takes in the worst case, and explain why.

(a) Consists of $2$ antichains $A$ and $B$, each with $5$ elements. 
Each of the $5$ elements of $A$ is known to be less than each of the
$5$ elements of $B$.
[[[Can draw diagram]]]

[[[I get 10 comparisons.  Can achieve by direct comparison of new key
with each element.  Doesn't seem we can better than, though -- every
key needs to be compared to $k$ directly or indirectly.  If we assume
$k$ is between all the elements of $A$ and all the elements of $B$,
then it seems we get no help from the existing edges.  Re-think this
-- if it looks too hard, we could default to the simpler case where we
are only allowed to compare $k$ to an element of the db.]]]

(b) Consists of $2$ chains, each of length $5$.  No relationships are
known between elements of the $2$ chains, however.
[[[Can draw diagram]]]

[[[Here I get $6$, by treating the two chains separately.]]]
