\documentclass[12pt,twoside]{article}


\usepackage{macros-course-lectures}

\def\given{\mid}
\def\period{.}

\parskip1.5ex
\parindent0pt



\begin{document}


%\let.=\period\par
%\futurelet\nextone.=\if \nextone\par\period\else\period\fi

%\def.{\futurelet\nextone\maybepar}

%\def\maybepar{\if\nextone\ \period\par\else\period\fi}


\def\outline{
%\catcode`\.=\active
\def\.{\ifmmode\period\else\period\par\fi}
}


\lecture{notes4}{Notes 4}{February 10, 2000}

\section{Induction and Recursive Algorithms}


\outline
  We've spent a lot of time studying induction\.  This is because
  induction comes up all the time in analyzing computation.  Why?
  Well, induction is a ``one step at a time'' proof method.
  Computations also evolve ``one step at a time''.


In fact, many of our inductive proofs can be seen as \emph{algorithms}
for solving a problem.  

We proved ``any $2^n$ by $2^n$ board can be tiled with Ls\period''  
If you look at the inductive proof, we actually gave a
  \emph{procedure} for carrying out that tiling.
Break the board in four, tile each piece, and merge the tilings.
What kind of algorithm is this? \emph{recursive:} it calls
 itself on smaller subproblems.

Similarly, the definition of the Fibonnacci numbers is also expressing
a recursive algorithm for computing them.  
And the inductive proof for Nim gave a game-winning algorithm.

Just about any inductive prove that a problem has a solution is in
fact providing a recursive algorithm for solving the problem.

Similarly, when you have a recursive algorithm that you think solves
some problem, and induction is the usual way to prove that it works.

In fact, an inductive proof can be thought of as a \emph{recursive
  algorithm} for writing down a proof:
\begin{structproof}
Procedure Write-Proof-For($n$)\\
\\
{\bf if} $n$ is 0\\
\> Write-the-base-case\\
{\bf else}\\
\> Write-Proof-For($n-1$)\\
\> Write-Inductive-Step($n$)\\
\end{structproof}
Assuming we have rule for writing down the base case $P(0)$, and
writing down the inductive step $P(n-1) \implies P(n)$ for any $n$, we
can write down a proof of $P(k)$ recursively: we recursively prove
$P(k-1)$, then we use the inductive step proof to prove
$P(k-1)\implies P(k)$.

\subsection{RSA}

RSA is at the core of many of the secure communication protocols we
use today (SSH, SSL, etc).  RSA works by treating any given message
(sequence of bits) as a big number $x$ and applying a mathematical
function that ``hides'' the number $x$ unless you know a secret key.

More precisely, it computes the remainder $r$ of $x^a$ on division by
$b$, where $a$ and $b$ are two specially constructed \emph{large}
numbers.  It turns out that if someone is given $r$ and knows the
right secret about the origin of $a$ and $b$, they can ``invert'' the
exponentiation and get $x$.

So, how can we compute the exponentiation?
The most obvious way is to work from the \emph{recursive definition}:
\begin{itemize}
\item $x^0=1$
\item $x^{n}=x\cdot x^{n-1}$
\end{itemize}
This is a definition of $x^n$, but also gives a recursive algorithm
for computing it:
\begin{structproof}
Procedure $\ms{Exp}(x,n)$\\
\\
{\bf if} $n=0$\\
\> return 1\\
{\bf else}\\
\> return $x*\ms{Exp}(x,n-1)$
\end{structproof}
Now it's pretty ``obvious'', but how would you \emph{prove} that
algorithm $\ms{Exp}$ returns the right answer?
Well, it doesn't always: what happens if you call $\ms{Exp}(5,-2)$?
$\ms{Exp}(8,2.5)$? 
(Trying to prove correctness can reveal implicit assumptions.  We
assume $n$ is a natural number).

\begin{theorem}
For all $x,n \in \nats$, a call to $\ms{Exp}(x,n)$ returns $x^n$
\end{theorem}
Note: the procedure $\ms{Exp}$ above can be thought of as (i) a
computational process or (ii) a recursive function definition (defined
as the output of the computational process).  We need to prove that it
defines the \emph{same} function as $x^n$.
We are actually claiming two distinct things in the theorem:
\begin{itemize}
\item The algorithm returns something (ie, doesn't run forever)
\item The algorithm returns the \emph{right} something.
\end{itemize}
\begin{proof}
By induction on $n$.  The base case, that $\ms{Exp}(x,0)$ returns
$x^0=1$, follows by inspection: on that call, the ``if'' condition is
satisfied, so we return 1.  For the inductive step, assume
$\ms{Exp}(x,n)$ returns $x^n$ and prove for $n+1$.  Consider the call
to $\ms{Exp}(x,n+1)$.  $n+1>0$, so the ``else'' clause is invoked.  We
call $\ms{Exp}(x,n)$ which, by assumption, returns $x^n$.  We multiply
that quantity by $x$, yielding $x^{n+1}$, and return it.  This proves
that $\ms{Exp}(x,n+1)$ returns the right value.
\end{proof}

OK, we've shown the procedure returns the right answer.
Is it a good implementation?
How many multiplications are done to compute $x^n$?  By induction,
there are $n$ multiplications.
Remember those ``128 bits keys'' we aren't allowed to export?  That
means the number $n$ has about 128 bits.
So we will be doing about $2^{128}$ multiplications.
Still be working on it when 6.042 finishes...

\subsection{Fast Exponentiation}

Let's cut down on the number of multiplies.
Suppose we want to compute $x^a$ and $a$ is even, say $2c$.
So we want to compute $x^a=x^2c$.
Any obvious way to beat $a$ multiplies?\\
Note $x^{2c}=(x^c)^2$\\
So compute $x^c$ ($c$ multiplies) and then square it (1 multiply).
Total: $c+1 =1+ a/2$ multiplies.
What if $a$ is odd?\\

Generalize this:

\begin{proofoverhead}
Procedure $\ms{FastExp}(x,n)$\\
\\
{\bf if} $n=0$\\
\> return 1\\
{\bf else}\\
\> {\bf if} $n$ is even\\
\> \> let $y=\ms{FastExp}(x,n/2)$\\
\> \> return $y^2$\\
\> {\bf else}\\
\> \> let $y=\ms{FastExp}(x,(n-1)/2)$\\
\> \> return $x*y^2$
\end{proofoverhead}

\begin{theorem}
For all $x,n \in \nats$, a call to $\ms{FastExp}(x,n)$ returns $x^n$
\end{theorem}
\begin{proof}
  Induction again.  Base case ($n=0$) follows by inspection as before.
  So let's prove the inductive step.
What kinds of subproblems are we going to use to prove it works for
  $n$?  {\bf not} just $n-1$, so let's use strong induction.
Suppose the theorem is true for every $k < n$, let's prove it for
  $n$\\
Suppose we call $\ms{FastExp}(x,n)$.  We can assume $n\ne 0$ (we
  already did a proof for 0) so the {\bf else} clause is selected in
  the algorithm.
Two cases.
If $n$ is even (say $n=2m$) then we  call
  $\ms{FastExp}(x,m)$.   By strong induction this returns $x^m$, which
  we assign to $y$.
Then we return $y=(x^m)^2=x^{n}$ as required, so the proof works for
  this case.
If $n$ is odd (say $n=2m+1$) then we call   $\ms{FastExp}(x,m)$ and
  set $y$ to the result.  
Then we return $x*y^2=x*(x^m)^2=x^{2m+1}=x^n$\\
So the proof works in this case too.
These are all the cases.  QED.
\end{proof}

How many multiplications does {\bf this} algorithm do?

\begin{theorem}
If $n$ has $b$ bits, then $\ms{FastExp}(x,n)$ performs at most $2b$
multiplications.  
\end{theorem}
\begin{proof}
Induction on $b$.
Base case:  If $b=0$, then $n=0$ and we return without doing
any multipliations.  So the bsae case is true.
Inductive step: assuming it true for $b$ bits, consider a $b+1$-bit
number $n$.
We execute a recursive call to   $\ms{FastExp}$ on either $n/2$
or $(n-1)/2$, depending whether $n$ is odd or even.
Both of these numbers have at most $b$ bits.
So by induction, the recursion does at most $2b$ multiplications.
Then we do either one more (even case) or two more (odd case). So at
most two more.
Overall, at most $2b+2 = 2(b+1)$ multiplications.  QED.
\end{proof}

Using this algorithm, a 128-bit key means only 256 multiplications:
quite an improvement!


\subsection{Greatest Common Diviser}


Given 2 positive natural numbers, $a, b \in N^+$, their \emph{greatest
  common divisor} is the largest number that divides both without
remainder.

(Recall notation: $a \mid b$ means $a$ divides $b$.)

Finding GCD is a problem that must be solved to generate the
``secret'' used to decrypt RSA messages.

One of the first recorded algorithms---Euclid's algorithm---finds
greatest common divisors.  Key lemma for ``simplifying'' problem:

\begin{lemma}
Given $x>y$, let $r=x-y$.  Then any common divisor of $x$ and $y$ is
also a common divisor of $y$ and $r$, and vice versa.
\end{lemma}
\begin{corollary}
 GCD$(x,y)=$GCD$(y,r)$
\end{corollary}
\begin{proof} (of the corollary).  $(x,y)$ and $(y,r)$ have exactly
  the same set of divisors (by lemma).  Thus, the biggest number in
  each set, namely the greatest common divisor, is the same.
\end{proof}
\begin{proof} (of the lemma).  Suppose $d \mid x$ and $d \mid y$.  So
  $x=zd$ and $y=wd$.  Thus $r=x-y=d(z-w)$ so $d \mid r$, meaning $d$
  is a common divisor of $y$ (by assumption) and $r$ (by deduction).
  The other direction of proof is similar.
\end{proof}

From this lemma we get Euclid's algorithm:
\begin{structproof}
Procedure $\ms{Euclid}(x,y)$\\
\\
{\bf if} $x=0$ return $y$\\
{\bf else if} $y=0$ return $x$\\
{\bf else if} $x > y$ return $\ms{Euclid}(y,x-y)$\\
{\bf else} return $\ms{Euclid}(x,y-x))$
\end{structproof}

Example: \\
Find $\ms{GCD}(112, 84)$ = ? \\
Step 1: $112 - 84 = 28$, reduce to $(28,84)$. \\
Step 2: $84 - 28 = 56$, reduce to $(28,56)$. \\
Step 3: $56 - 28 = 28$, reduce to $(28,28)$. \\
Step 4: $28 - 28 = 0$, reduce to $28,0$\\
Now done, answer is $28$. \\
\\

Let's prove this algorithm does the right thing.
First faint worry: maybe we've got an infinite recursion?\\
First thing to prove for any algorithm: it \emph{terminates}

\begin{lemma}
For any natural number inputs $x$ and $y$, $\ms{Euclid}(x,y)$ returns
a value.
\end{lemma}
Why do we think this is true intuitively?  Because the arguments
``shrink'' on each recursive call and eventually reach 0.  Induction
lets us formalize this.
\begin{proof}
  Induction on $x+y$.
  More formally, we prove by strong induction on
  $n$ that for all inputs $x$ and $y$ such that $x+y=n$, the algorithm
  terminates.
Base case: if $x+y=0$, then both $x$ and $y$ are 0 so we return
  immediately.
Inductive step.  Assume the claim for $x+y<n$ and prove it for
  $x+y=n$.  
Now consider two cases.  Either $x \ge y$ or $y \ge x$.  Suppose first
  that $x \ge y$.
So consider two subcases.  If $y=0$, then we return immediately and the
  inductive step is proved.  If $y > 0$, then we make a recursive call
  with arguments $y$ and $x-y < x-0 =x$.
So the sum of the two new arguments is less than $x+y=n$\\
So the inductive hypothesis holds: the recursive call returns a
  value.
So our original call returns (the same) value.

What about the second case ($y \ge x$)?  Well, I would have to write
the same proof all over again, swapping $x$ and $y$ everywhere.  To
avoid such tedium, we say ``without loss of generality, assume $x \ge
y$.  This is code for ``the other case works exactly the same way''
and completes the proof.
\end{proof}

\begin{lemma}
For any natural number inputs $x$ and $y$, $\ms{Euclid}(x,y)$ returns
GCD$(x,y)$
\end{lemma}
\begin{proof}
Induction on $x+y$ as before.  The base case is trivial (assuming we
define GCD$(0,0)$ to be 0, which seems fair).

For the inductive step where $x+y= n > 0$, assume without loss of
generality that $x \ge y$.  Then we return the result of a recursive
call on $\ms{Euclid}(y,x-y)$.  This means (by strong induction) that
we return GCD$(y,x-y)$.  But we proved a lemma above that
GCD$(x,y)=$GCD$(y,x-y)$ so we are returning the right value.  
\end{proof}

Wait a minute, what's with the GCD$(0,0)$ definition?  Well,
technically every number is a common divisor of 0 and 0, so it should
be infinite?  

Why doesn't this matter?  Because we never actually make a recursive
call to $(0,0)$---one one number is 0, we terminate.

So even though we still need a base case for the induction pattern, we
never actually apply that base case.  Instead our base case is
implicit in the ``subcase'' of the inductive step where on argument is 0.

\section{Sets}

Today we give an overview of set theory, including relations on sets. \\
Probably much of the first part of this will be review from high
school math. \\
\\
We will define an interesting kind of relation called an {\em
equivalence relation\/}. \\
\\
An important example of equivalence relation is 
number equivalence mod $m$, where $m$ is a positive integer. 
We will define this relation and give some of its properties. \\
We will conclude with an application of modular equivalence 
ideas to get fast RSA encryption and decryption.


\subsection{Definitions}

A {\em set} is a collection of objects. \\
(Collection, object are undefined terms.) \\
\\
The order of elements is not important. I.e., the sets 
$\{1,2\}$ and $\{2,1\}$ are considered the same set.
Each element is included only once. So, if we write
$\{a,b\}$ or $\{a,a,b\}$ they both denote the same set
and the element $a$ is considered to be present in the set 
only once. (repeating an element in the description of 
a set may be useful sometimes).
\\
Examples of sets: 

\begin{itemize}
\item $\{1,2,3\}$. 

A simple finite set, obtained by enumerating all the elements.

\item $\emptyset$, the {\em empty set}.

A special case of a finite set, with {\em no\/} elements.

\item $\{\emptyset\}$.

This funny set is different from $\emptyset$. \\
It has one element, which is a set.

\item $\{1,\{1,2\}\}$.

This one contains two elements, one of which is a set itself. \\
The set happens to contain the other element $1$, but that does not
contradict the condition that an element is included only once. 

\item $\{n\in \naturals  \given n \hbox{ is even }\}$.

An infinite set, specified by giving a universal set (domain of
discourse) and a property. 
In general, given a domain of discourse $D$ and a predicate $P$
over $D$, $\{x\in D \mid P(x)\}$ denotes the set of all 
$x$ in $D$ that satisfy predicate $P$.
\end{itemize} 

We write that $a\in A$ (``$a$ is in $A$'' or ``$a$ is an element of
$A$'') if $a$ is an object in the set $A$. \\
\\
The empty set satisfies the proposition $(\forall x) x \notin \emptyset$. \\
\\
Sets can be defined by axioms and deduction rules, giving a very
formal {\em axiomatic set theory\/}. \\
But usually, people reason about sets less formally, using {\em naive
set theory\/}. \\
Either way, set theory can be used as the basis for defining almost
everything else in mathematics and is therefore
regarded as the most fundamental part of math. \\
\\
E.g., natural numbers can be defined as certain kinds of sets (!).
For example we can represent the number $0$ by the empty set $\emptyset$,  
number $1$ by the set $\{\emptyset\}$, number $2$
by $\{\emptyset,\{\emptyset\}\}$, and so on.
In general, if $S_n$ is the set representing the number $n$,
the successor of $n$ can be represented by the set
$S_{n+1} = \{\emptyset\}\cup \{\{x\}\mid x\in S_n\}$.
Then the properties natural numbers can be {\em proved as
theorems\/} of set theory, 
rather than being assumed. \\
\\
Some typical axioms of axiomatic set theory: \\
(Seem ridiculously obvious but aren't)

\begin{itemize}
\item
The {\em axiom of extensionality} says that two sets are equal iff they have
the same elements. \\
\\
E.g., the two sets $\{n\in \naturals  \mid 2 \mbox{ divides } n\}$ and
$\{2n \mid n \in \naturals\}$ \\
are defined in different ways. \\
But they are ``equal'' as sets because they have exactly the same
elements. \\

\item
The {\em axiom of comprehension\/} says that we can form a new set by including
all objects of a given set that share a common property. \\
\\
For example, 
$\{x \in {\Bbb R} \given P(x)\} \hbox{  where  } P(x)=\hbox{ ``x is
rational''}$ denotes the set of all rational numbers.
\\
The axioms says that this is a set. \\
\end{itemize}

The set building notation that we used above has given the universe of
discourse explicitly.  \\
Sometimes, when this seems obvious, we will leave this out. \\
Just like what we did for predicate quantifiers. \\
E.g., we might write:
$\{n \mid n \hbox{ is even }\}$
if the universe $\naturals$ seems clear. \\
(Could be confusing -- the universe could alternatively be the
integers, positive and negative.)

\begin{defin}
We say that $A\subseteq B$ (``$A$ is a subset of $B$'' or 
``$A$ is contained in $B$'') if every
element of $A$ is also an element of $B$ (that is, $(\forall x)(x\in
A \implies x \in B)$).
\end{defin}

\begin{defin}
 We say that $A$ is a {\em proper} subset of $B$
($A\subset B$) if $A\subseteq B$ and $A\neq B$.
\end{defin}

Note $(\forall X) \emptyset \subseteq X$, i.e. the empty set 
is contained in any other set. \\
\\
Q: Can we have both $A\in B$ and $A\subseteq B$? \\
\\
A: Yes, consider $\{ 1, \{ 1 \} \}$.

\begin{theorem}
$A=B$ if and only if $A \subseteq B$ and $B \subseteq A$.
\end{theorem}

Proof by axiom of extensionality.  Ignore here. \\
\\
This observation is very important as it provides a simple way to
prove that 2 sets are equal. \\
\begin{example}
Prove that the two sets $\{n\in \naturals  \mid 2 \mbox{ divides } n\}$ and
$\{2n \mid n \in \naturals\}$ are equal. \\
\\
Must show containment in both ways. \\
$\subseteq$: \\
We assume $x \in \naturals$ and $2 \mbox{ divides } n$, and show that
there is some $y \in \naturals$ such that $x = 2y$. \\
This is true by definition of divisibility. \\
$\supseteq$: \\
Assume $x = 2y$.  Then $x$ is divisible by $2$, again by definition
of divisibility. 
\end{example}
\begin{example}
Slightly more interestingly: \\
\\
The set of evens $\geq 2$, i.e.,
$\{ 2n \mid n \in \naturals, n \geq 1 \}$, \\
is equal to 
$\{ x \in \naturals \mid \exists y, z \mbox{ odd }, x = y + z \}$.\\
$\subseteq$: \\
If $x = 2n$, then write $x = 1 + (x - 1)$, sum of two odds. \\
$\supseteq$: \\
If $x = y + z$, where both $y$ and $z$ are odd, then write $y = 2a +
1$, $z = 2b + 1$. \\
Then $x = 2(a+b+1)$, which is even.
\end{example}

The size of a set $A$ (the number of elements it contains) is written
$|A|$.  \\
For example, the set $\{a,b,c\}$ has size 
$|\{a,b,c\}|=3$. The empty set has size 
$|\emptyset|=0$.\\
The size of a  finite set is a natural number. \\
If $A$ is an infinite set, we write $|A|=\infty$.

Notice that for any two finite sets $A,B$, we have 
$|A\cup B| \leq |A|+|B|$ and equality holds if and only if 
$A\cap B=\emptyset$.
If we allow $A$ and $B$ to be infinite sets,
we still have $|A\cup B| \leq |A|+|B|$
if we define $x+\infty = \infty+x=\infty$ and 
$x\leq \infty$ for all $x\in \naturals$ and $x=\infty$.

\subsection{Russell's paradox}

We might think that set theory is obvious, but this is not the case. \\
Consider the set $S = \{x \mid x \mbox{ is a set}\}$.  \\
This seems like a perfectly reasonable set (well, maybe a bit
strange---note $S \in S$). \\
If our set theory allows for such an entity, we get into trouble as
follows. \\
\\
Consider the set $Q=\{x\in S \given x \hbox{ is a set and }x\not\in x\}$. \\
\\
If $S$ is a set, then so is $Q$ by standard axioms like
comprehension. \\
Note $Q \subseteq S$.  \\
\\
Then some sets are in $Q$ and some aren't. \\
\\
Example of something in $Q$: \\
The set $A = \{1,2,3\}$ is not an element of itself, so $A \in Q$. \\
\\
Example of something that is not in $Q$: \\
The set 
$P = \{\hbox{ all infinite sets}\}$ is a set (by comprehension from $S$). \\
Therefore, $P \in P$ as the set $P$ itself is infinite. \\
So $P \not \in Q$. \\
\\
The troublesome question is: is $Q\in Q$? 
\begin{enumerate}
\item If $Q \in Q$ then $Q \not \in Q$, a contradiction.
\item If $Q \not \in Q$ then $Q \in Q$ also a contradiction.
\end{enumerate} 

This is clearly a paradox. \\
In fact, this paradox, at the time it was discovered by Russell
($\sim1900$), was a blow to the then current attempts to formalize all
of mathematics axiomatically. \\
\\
Mathematicians had to work very hard to find a consistent set of axioms
(note that a consistent set of axioms does not allow this paradox to
arise). \\
\\
The problem with naive set theory is that you can form a set
by describing its members, as we did for $Q$ above.  \\
Axiomatic set theory has a specific set of rules for how you can form
sets---rules that don't allow the set $Q$ above to be specified, and
therefore avoid the paradox. \\
\\
Fortunately, we'll be working naively...

\subsection{Basic set operations}

There are several standard operations on sets. \\
In axiomatic set theory, the claims that these operations yield
new sets are axioms.

\begin{enumerate}
\item Union

$A\cup B=\{x\given x\in A \hbox{  or  } x\in B\}$. 

\item Intersection 

$A\cap B=\{x\given x\in A \hbox{  and  } x\in B\}$. 

\item Difference

$A-B=\{x\given x\in A \hbox{  and  } x\not\in B\}$. 

Also written as $A\setminus B$.

\item Complement

$A^c=\{x\given x\not\in A\}$. 

Also written as $\bar A$.

\item Symmetric difference

$A \oplus B= A \cup B - A\cap B$.
\end{enumerate} 

Sets and operations on sets can be represented using Venn diagrams
(see textbook, sections 1.4, 1.5). \\
\\
Union and intersection can be extended to apply to more than two sets.  \\
Given a finite collection of sets $A_i$, $i=1,\ldots,n$,
\[
\bigcup_{i=1}^n A_i
\]
is the set of elements in at least one of the $A_i$, and  \\
\\
\[
\bigcap_{i=1}^n A_i
\]
is the set of elements in all of the $A_i$.
Both of these make sense if $i=1$ -- reduce to just $A_1$. \\
In fact, for $i = 0$, default says union of no sets is $\emptyset$ and
intersection of no sets is universal set. \\
This definition of union and intersection of no sets might seem
arbitrary, but in fact it can be motivated in several ways.
For example, if we want to define the union of a finite 
family of sets $A_1,A_2,\ldots, A_n$, we can define
$\bigcup_{i=1}^{1} A_1 = A_1$ and 
$\bigcup_{i=1}^{n+1}A_i = (\bigcup_{i=1}^{n} A_i)\cup A_{n+1}$.
If we want this recursive definition to work also for base case 
$n=0$ we must set $\bigcup \emptyset = \bigcup_{i=1}^0 A_i = \emptyset$.
Analogously, for intersection. The recursive definition 
of  $\bigcap_{i=1}^{n+1}A_i = (\bigcap_{i=1}^{n} A_i)\cap A_{n+1}$
must have base case 
$\bigcap \emptyset = \bigcap_{i=1}^0 A_i = U$
if we want $\bigcap \{A_1\} = (\bigcap \emptyset)\cap A_{1} = A_1$.
\\

Instead of using explicit index, can just consider a set of sets:
If ${\cal S}$ is a {\em set of sets}, then 
\[
\bigcup {\cal S}
\] 
denotes the union of all the sets in ${\cal S}$, in any order. \\
(Can prove that the order doesn't matter, will do below.) \\
Likewise, 
\[
\bigcap {\cal S} 
\]
denotes the intersection of all the sets in ${\cal S}$. \\
\\
Complement is different from all the other operations, because it
assumes some universe $U$ from which we extract the things not in $A$.  \\
All sets in question are supposed to be subsets of $U$. \\
Given a universal set $U$, the complement operation 
can be expressed as $A^c = U - A$. \\
The universe $U$ is often implicit, where there seems to be little
chance of confusion. \\
\\
There is a close correspondence between set operations and logical operations:
consider sets $A$, $B$ and $C$ defined by 
$A=\{x\mid P(x)\}$, $B=\{x\mid Q(x)\}$, $C=\{x\mid R(x)\}$, 
where $P,Q,R$ are predicates. \\
\begin{itemize}
\item set union vs. predicate or: 
 $A\cup B =  C$ if and only if 
 $P(x)\vee  Q(x) \equiv  R(x)$ \\
\item set intersection vs. predicate and: 
 $A\cap B =  C$ if and only if 
 $P(x) \wedge Q(x)\equiv R(x)$ \\
\item set difference vs. predicate ``non implication'':
 $A - B = C$ if and only if 
 $\neg (P(x)\rightarrow Q(x)) \equiv R(x)$\\ 
\item set complement vs. predicate not:
 $A^c=B$ if and only if $\neg P(x) \equiv Q(x)$\\
\item set symmetric difference vs. predicate exclusive or:
 $A\oplus B = C$ if and only if $P(x)\oplus Q(x)\equiv R(x)$.
\end{itemize}

Predicates and sets often form alternative ways of looking at the same
notions: \\
Given predicates $p$ and $q$, define $P = \{ x \mid p(x) \}$, \\
$Q = \{ x \mid q(x) \}$. \\
Then $P \cup Q = \{ x \mid p(x) \vee q(x) \}$. \\
Etc.
\\
\\
\subsection{Power set (set of all subsets)} 

Another very important operation on sets is that of the {\em power
set} of a given set $A$. \\
It is denoted by ${\cal P}(A)$ and consists of the set of all subsets
of $A$. \\
In other words, ${\cal P}(A) = \{x\given x\subseteq A\}$. \\
\\
In particular, for any set $A$,
$\emptyset\in{\cal P}(A)$ and $A\in {\cal P}(A)$.
\\
Ex: \\
If $A=\{1,2\}$ then ${\cal P}(A)=\{\emptyset,\{1\},\{2\},\{1,2\}\}$.

\begin{theorem}
If $A$ has $n$ elements, then $P(A)$ has $2^n$ elements.
\end{theorem}

\begin{proof}
Ordinary Induction on $n$. 
BASE:  $n = 0$.  Then $P(A)$ has one element, $\emptyset$. \\
INDUCTIVE: Assume for $n$, prove for $n+1$. \\
Pick one element, $a$. \\
The subsets can be divided into two groups -- those containing $a$ and
those not containing $a$. \\
Moreover, the set of subsets of $A$ not containing $a$ is 
${\cal P}(A-\{a\})$, and the set of subsets of $A$ containing $a$
is the set $\{\{a\}\cup B \mid B\in {\cal P}(A-\{a\})$ which has 
the same number of elements as ${\cal P}(A-\{a\})$.
By induction hypothesis ${\cal P}(A-\{a\})$ has $2^{n-1}$
elements,
so the total is $2^{n-1} + 2^{n-1} = 2^n$.
\end{proof}

Q: I did a base case of $0$ here.  Does the inductive step from $0$ to
$1$ work correctly? \\
\\
A: Yes, it corresponds to the sets $\{\emptyset\}$ and $\{ a \}$. \\
\\
Because of this cardinality relationship, 
${\cal P}(A)$ is sometimes denoted by $2^A$.

\subsection{Set identities}

There are a large number of set identities involving relationships
among sets built using various operations. \\
See p. 50 of book. \\
\\
Here we just list some sample properties, and show how they can be
proved.

\begin{theorem}
(Distributivity:  Union distributes over intersection) \\
$A \cup (B \cap C) = (A \cup B) \cap (A \cup C)$
\end{theorem}

\begin{proof}
We show the two sets are equal by proving that each is contained in the
other. \\
\begin{enumerate}
\item
$A \cup (B \cap C) \subseteq (A \cup B) \cap (A \cup C)$. 

Suppose $x \in A \cup (B \cap C)$.  \\
Then by definition, $x \in A$ or $x \in (B \cap C)$.  \\
We argue by cases.  \\
\\
If $x \in A$, then by definition $x \in A \cup B$ and similarly $x \in
A \cup C$.  \\
It follows by definition of intersection that $x$ is in the intersection
of these two sets, which is the RHS. \\
\\
On the other hand, if $x \in (B \cap C)$, then $x \in B$ and $x \in C$. \\
Then by definition of union, $x \in A \cup B$ and $x \in A \cup C$. \\
It again follows that $x$ is in the intersection. 
\\
\item
$A \cup (B \cap C) \supseteq (A \cup B) \cap (A \cup C)$. \\
\\
Suppose $x \in (A \cup B) \cap (A \cup C)$. \\
It follows that $x \in A\cup B$ and $x \in A \cup C$. \\
We argue by cases. \\
\\
If $x \in A$, then certainly $x \in A \cup (B \cap C)$. \\
\\
If $x \notin A$, then since $x \in A \cup B$, we know $x \in B$. \\
Similarly, $x \in C$. \\
Therefore, $x \in B \cap C$. \\
It follows that $x \in A \cup (B \cap C)$. 
\end{enumerate}
\end{proof}


\begin{theorem}
De Morgan's rule: $(A\cup B)^c=A^c\cap B^c$.
\end{theorem} 
\begin{proof}
We have to show that $(A\cup B)^c \subseteq A^c \cap B^c$ 
and that $A^c \cap B^c \subseteq (A\cup B)^c$. 

\begin{enumerate}
\item
$(A\cup B)^c\subseteq A^c\cap B^c$.

Let $x\in (A\cup B)^c$. 
Then by definition of complement, $x \notin A \cup B$.  
It follows (proof by contradiction) that $x \notin A$ and $x \notin B$.  
Therefore, again by definition of complement, $x \in A^c$ and $x \in B^c$.  \\
Then by definition of intersection, $x \in A^c \cap B^c$.

\item
$A^c\cap B^c\subseteq (A\cup B)^c$.

Let $x \in A^c \cap B^c$. \\
Then $x$ is in $A^c$ and in $B^c$. \\
Therefore, $x$ is in neither $A$ nor $B$ and therefore not in $A \cup B$. \\
We conclude that $x \in (A \cup B)^c$.
\end{enumerate} 
\end{proof} 

The above two theorems have {\em dual theorems} produced by swapping
$\cap$ and $\cup$. \\
This is a general phenomenon -- look at p. 50, arranged in pairs. \\
Duality can be applied to any theorem that is true for {\em all} sets.
Each theorem in a pair can be obtained from the other by 
swapping the operations $\cap$ and $\cup$, 
and the sets $\emptyset$ and $U$.
Notice that complement operations are not changed.

\begin{theorem}
$A \cap (B \cup C) = (A \cap B) \cup (A \cap C)$
\end{theorem}

\begin{theorem}
 $(A\cap B)^c=A^c\cup B^c.$
\end{theorem}

\subsection{Cartesian products}

Another important set operation is the {\em Cartesian product}. \\
For sets $A$ and $B$ define $A\times B=\{(a,b)\given a\in A\hbox{  and
}b\in B\}$. \\
Here, $(a,b)$ is considered an {\em ordered} pair so that $(a,b)\neq (b,a)$.

\begin{example}
If $A=\{0,1\}$ and $B=\{x,y\}$ then $A\times
B=\{(0,x),(0,y),(1,x),(1,y)\}$.
\end{example}

Order is important!  \\
In the above example, $(a,0) \notin A \times B$, though $(a,0) \in B
\times A$. \\
\\
We can generalize Cartesian products to apply to more than 2 sets and define 
$A_1 \times \cdots \times A_n 
= \prod_{i=1}^n A_i 
= \{(a_1,\cdots,a_n) \given a_i\in A_i \hbox{ for   }1\le i \le n\}$.
Also, we write $A^n$ for $\prod_{i=1}^n A$.

Examples: 
\begin{itemize}
\item $\{0,1\}^n$ is the set of all $n$-tuples of 0's and 1's. 
Or, length $n$ strings of $0$'s and $1$'s.
\item $\Re^2 = \Re\times \Re$ is the real plane.
\end{itemize} 

\section{Relations}

A ``relation'' is a fundamental mathematical notion expressing a
relationship between elements of sets. \\
It's an abstract notion useful in practice for modelling many
different sorts of relationships. \\
It's the basis of the {\em relational data base\/} model, the standard
data model for practical data processing systems.

\subsection{Definitions}

A {\em binary relation} {\em from} 
a set $A$ {\em to} a set $B$ is a subset 
$R\subseteq A\times B$. \\
A binary relation {\em on} a set $A$ is a subset $R\subseteq A^2$
(I.e., a relation from $A$ to $A$).\\
We often write $a\sim_R b$ or $aRb$ instead of  $(a,b)\in R$. \\
\\
We can also define a {\em ternary} relation on $A$ as a subset
$R\subseteq A^3$ or, in general, an $n$-ary relation as a subset
$R\subseteq A^n$, or $R\subseteq A_1\times A_2 \times\cdots\times A_n$
if the sets $A_i$ are different. \\
\\
Relations are used to model many different things:

\begin{enumerate}
\item \label{classes} 
The relation ``is taking class'' as a subset of
$\{$students at MIT$\}\times\{$classes at MIT$\}$. 
A relation from students to classes.
\item \label{roommates} 
The relation ``is living in the same room'' as a subset of
$\{$students at MIT$\}\times \{$students at MIT$\}$.
A relation on students.
\item \label{reachable} 
The relation ``can drive from first to second city''.
(Not necessarily directly -- just some way, on some roads.)
\item \label{wire} 
Relation on computers, ``are connected (directly) by a wire''
\item \label{meet} ``meet one another on a given day''
\item \label{likes}  ``likes''
\item \label{le} 
Let $A=\naturals$ and define $a\sim_R b$ iff $a\le b$.
\item \label{finite} 
Let $A={\cal P}(\naturals)$ and define $a\sim_R b$ iff $|a \cap b|$ is finite.
\item \label{dist} Let $A=\Re^2$ and define $a\sim_R b$ iff $d(a,b)=1$.
\item \label{contain} Let $A={\cal P}(\{1,\cdots,n\})$ and let $a\sim_R b$ if
$a\subseteq b$.
\end{enumerate} 

\subsection{Properties of Relations}

There are several standard properties that are satisfied by (some)
relations. \\
These occur commonly enough that they are worth identifying. \\
Sometimes interesting things can be deduced just from the abstract
properties. \\

A binary relation $R$ on $A$ is:
\begin{enumerate}
\item {\em reflexive} if for every $a\in A$, $a\sim_R a$.
\item {\em symmetric} if for every $a,b\in A$, $a\sim_R b$ implies $b\sim_R a$.
\item {\em antisymmetric} if for every $a,b\in A$, $a\sim_R b$ and
$b\sim_R a$ implies $a=b$.
\item {\em transitive} if for every $a,b,c\in A$, $a\sim_R b$ and $b\sim_R c$
implies $a\sim_R c$. 
\item {\em asymmetric} if for every $a,b\in A$, $a\sim_R b$ implies 
$\neg(b\sim_R a)$.
\end{enumerate} 

The difference between antisymmetric and asymmetric relations, 
is that antisymmetric relations may contain pairs $(a,a)$,
i.e., elements can be in relations with themselves,
while in an asymmetric relation this is not allowed.
Clearly, any asymmetric relation is also antisymmetric,
but not vice versa.

Among our examples:
\begin{itemize}
\item Relation~\ref{roommates} is reflexive, symmetric, transitive. 
\item Relation~\ref{reachable} is reflexive, transitive.  \\
Not necessarily symmetric, since roads could be one-way, but in
actuality...
But definitely not antisymmetric.
\item Relation~\ref{wire} is symmetric but not transitive. \\
Whether it is reflexive is open to interpretation.
\item Relation~\ref{meet} likewise. 
\item Relation~\ref{likes} is (unfortunately) not symmetric.  Not
antisymmetric.  Not transitive.  Not even reflexive!
\item Relation~\ref{le} is reflexive, antisymmetric, transitive.
\item Relation~\ref{finite} is not reflexive.  It is symmetric.  Not
transitive.
\item Relation~\ref{dist} is only symmetric.
\item Relation~\ref{contain} is reflexive, antisymmetric and transitive.
\end{itemize} 

\subsection{Representation}

There are different ways of representing relations, e.g., for handling
in a computer program. \\
\\
We can describe by properties, as we did above. \\
That's about all we can do, for infinite sets. \\
But for finite sets, usually we use some method that explicitly
enumerates all the elements of the relation. \\
\\
Some alternatives: Lists, Matrices, Graphs. 

\subsection{Lists}

To represent a finite relation from set $A$ to set $B$, just
list all the pairs. \\
\\
Ex: Consider for example the relation from 
$A = \{ 0, 1, 2, 3 \}$ to $\{ a,b,c \}$ defined by the list 
$\{ (0,a), (0, c), (1, c), (2, b), (1,a) \}$. \\

Ex: The divisibility relation on natural numbers $\{ 1, \cdots, 12 \}$
is represented by the list: \\
$\{ (1,1), (1,2), \ldots, (1,12), (2,2), (2,4), \ldots, (2,12), $\\
$ (3,3), (3,6), (3,9), (3,12), (4,4), (4,8), (4,12), (5,5), (6,6), $\\
$(6,12), (7,7), (8,8), (9,9), (10,10), (11,11), (12,12) \}$. \\
\\

Reflexivity in this representation: Contains all pairs $(a,a)$. \\
Symmetry: If contains $(a,b)$ then contains $(b,a)$. \\
Transitivity: If contains $(a,b)$ and $(b,c)$ then contains $(a,c)$. \\
\\

\subsection{Boolean matrices}

Used for finite sets $A$, $B$ \\
Rows for elements of $A$, columns for $B$. \\
$1$ in position if the pair is in the relation, $0$ otherwise.

Ex: The relation from
$\{0,1,2,3\}$ to $\{a,b,c\}$ of our first example is 
represented by the matrix
\[\begin{array} {r|ccc}
 &a& b&  c \\ 
\hline
0&1& 0&  1\\
1&1& 0&  1\\
2&0& 1&  0\\
3&0& 0&  0\\
\end{array}\]

Ex: The divisibility relation over $\{1,2,\ldots,12\}$ is 
represented by the matrix

\[\begin{array}{r|cccccccccccc}
  &1  &2  &3  &4  &5  &6  &7  &8  &9 &10 &11 &12\\
\hline
1 &1  &1  &1  &1  &1  &1  &1  &1  &1  &1  &1  &1\\
2 &0  &1  &0  &1  &0  &1  &0  &1  &0  &1  &0  &1\\
3 &0  &0  &1  &0  &0  &1  &0  &0  &1  &0  &0  &1\\
4 &0  &0  &0  &1  &0  &0  &0  &1  &0  &0  &0  &1\\
5 &0  &0  &0  &0  &1  &0  &0  &0  &0  &1  &0  &0\\
6 &0  &0  &0  &0  &0  &1  &0  &0  &0  &0  &0  &1\\
7 &0  &0  &0  &0  &0  &0  &1  &0  &0  &0  &0  &0\\
8 &0  &0  &0  &0  &0  &0  &0  &1  &0  &0  &0  &0\\
9 &0  &0  &0  &0  &0  &0  &0  &0  &1  &0  &0  &0\\
10&0  &0  &0  &0  &0  &0  &0  &0  &0  &1  &0  &0\\
11&0  &0  &0  &0  &0  &0  &0  &0  &0  &0  &1  &0\\
12&0  &0  &0  &0  &0  &0  &0  &0  &0  &0  &0  &1\\
\end{array}\]

Reflexivity in this representation:  major diagonal is all 1 \\
Symmetry: the matrix is symmetric around major diagonal \\
Transitivity: ... \\
\\
\\
\subsection{Digraphs} 

Digraphs are used for relations where $A = B$, 
i.e., for relations on a finite set $A$.
To represent a relation as a digraph we draw a dot (vertex) 
for each element of $A$, and draw an arrow from first element
to second element of each pair in the relation. \\
The digraph may contain self-loops, i.e. arrows from a dot 
to itself, associated to the elements $a$ such that $(a,a)$ is 
in the relation. \\

The relation from
$\{0,1,2,3\}$ to $\{a,b,c\}$ of the previous examples 
cannot be represented as a digraph because $A\neq B$.

The divisibility relation over $\{1,2,\ldots,12\}$
is represented by the digraph
\begin{center}
\mbox{\psfig{figure=dividehasse.ps}}
\end{center}

Reflexivity: All nodes have self-loops. \\
Symmetry: all edges are bidirectional. \\
Transitivity: Short-circuits, for any sequence of consecutive arrows,
there is a single arrow from the first to the last node. 

\subsection{Composition}

 
Notice the difference between properties of relations 
and operations on relations. A property, such as 
symmetry or transitivity, is something a relation 
may or may not satisfy. An operation on relation, 
such as relation composition we are going to define, 
is a way to combine one or more relations to build 
some other relation.
\\

The {\em composition} of relations $R_1 \subseteq A \times B$ and $R_2
\subseteq B \times C$ is the relation $R_2 \circ R_1 =\{(a,c) \mid
(\exists b) ((a,b) \in R_1) \wedge ((b,c) \in R_2)\}$. \\
Notice that $R_1\circ R_2$ and $R_2\circ R_1$ are different.
This is the notation used in the text book. 
Notice that some people prefer to write $R_1\circ R_2$ for 
the composition of $R_1$ and $R_2$, which may seem 
more natural,
but from now on we will stick with the book.
\\
Examples: \\
Composition of parent-of relation with itself gives grandparent-of. \\
Composition of child-of relation with parent-of relation gives
sibling-of relation. \\
\\
\\
Composition of relations is equivalent to matrix multiplication, with
$+$ replaced by $\vee$ (Boolean or) and $*$ replaced by $\wedge$. \\
\\
Ex: \\
Let $R_1$ be relation from first example above: \\
\[\begin{array}{r|ccc}
 & a&  b&  c\\
\hline  
0& 1&  0&  1\\
1& 1&  0&  1\\
2& 0&  1&  0\\
3& 0&  0&  0\\
  \end{array}
\]

Let $R_2$ be relation from $\{ a, b, c \}$ to $\{ d,e,f \}$ given by:
\[\begin{array}{r|ccc}
 & d&  e&  f\\
\hline  
a& 1&  1&  1\\
b& 0&  1&  0\\
c& 0&  0&  1\\
  \end{array}
\]

Then $R_2 \circ R_1 = \{ (0,d), ... \}$, 
that is, \\
\[\begin{array}{r|ccc}
 & d&  e&  f\\
\hline  
0& 1&  1&  1\\
1& 1&  1&  1\\
2& 0&  1&  0\\
3& 0&  0&  0\\
  \end{array}
\]

Same as matrix multiply, using Boolean ops and and or. \\
\\
\\
Relational composition is associative. \\
Just like matrix multiplication. \\

\begin{lemma}
\label{associative}
$R_1 \circ (R_2 \circ R_3) = (R_1 \circ R_2) \circ R_3$
\end{lemma}

The composition $R \circ R$ of $R$ with itself is written $R^2$. \\
Similarly $R^n$ denotes $R$ composed with itself $n$ times. \\
Recursive definition: \\
$R_1 = R$, $R^n = R \circ R^{n-1}$. \\
Actually, can define $R_0 = \{ (a,a) \mid a \in R \}$ as base case. \\
We'll just use $1$ as base case, as book does. \\
\\
Could just as well have defined as $R^{n-1} \circ R$:

\begin{theorem}
For $n \geq 2$, $R^n = R^{n-1} \circ R$
\end{theorem}

\begin{proof}
Prove by induction on $n$.  \\
Base case $n=2$ trivial. \\
Inductive step assumes $n \geq 3$. \\
$R^n = R \circ R^{n-1}$ by definition. \\
By inductive hypothesis, this is equal to $R \circ (R^{n-2} \circ R)$. \\
By associativity, Lemma \ref{associative},
this is equal to $(R \circ R^{n-2}) \circ R$. \\
By definition of $R^{n-1}$, this is equal to $R^{n-1} \circ R$. \\
As needed.
\end{proof}

The following gives simple conditions for transitivity.

\begin{theorem}
$R$ is transitive if and only if 
$R^2 \subseteq R$ if and only if
$R^n \subseteq R$ for all positive integers $n$.
\end{theorem}

\begin{proof}
Obviously, the third implies the second. \\
\\
Show second implies the first: \\
To show transitivity, assume $(a,b) \in R$ and $(b,c) \in R$. \\
Then $(a,c) \in R^2$. \\
Second says $(a,c) \in R$, which implies transitivity. \\
\\
Show first implies third. \\
Use induction on $n$. \\
BASE:  $n = 1$, $R \subseteq R$, obvious. \\
INDUCTIVE STEP: \\
Suppose $R^n \subseteq R$, show $R^{n+1} \subseteq R$. \\
Say $(a,b) \in R^{n+1}$.  \\
Then for some $c$, $(a,c) \in R^n$ and $(c,b) \in R$ for some $c$.  \\
(Uses theorem above.) \\
Then $(a,c) \in R$ by induction. \\
Then by transitivity, $(a,b) \in R$.
\end{proof}

In the above theorem we wanted to prove the equivalence of three 
propositions $A\leftrightarrow B \leftrightarrow C$.
We proved it by showing that the three implications
$A\rightarrow B$, $B\rightarrow C$ and $C\rightarrow A$
holds true.
Why is this enough to prove equivalence of $A$, $B$ and $C$?
Notice that from $B\rightarrow C \rightarrow A$ we can deduce 
by transitivity $B\rightarrow A$, which together with 
$A\rightarrow B$ proves that $A\leftrightarrow B$.
Equivalences $B\leftrightarrow C$ and 
$C\leftrightarrow A$ can be derived similarly. 


\section{Equivalence relations and partitions}

Now we consider an important special type of relation, called an {\em
equivalence relation}. \\
We give an important special case -- modular arithmetic.  \\
And an application of this to RSA.

\subsection{Definitions}

An {\em equivalence relation} is a relation that is reflexive,
symmetric and transitive. \\
For example, the ``roommates'' relation is an equivalence relation. \\
Trivial example: ``='' relation on natural numbers. \\
Hallmark of equivalence relations is the word ``same''. \\
\\
Equivalence relations partition the universe into subsets in a natural
way: \\
\\
A {\em partition} of a set $A$ is a collection of subsets
$\{A_1,\cdots,A_k\}$ such that any two of them are disjoint (for any
$i\neq j$, $A_i\cap A_j=\emptyset$) and such
that their union is $A$. \\
That is, every element of $A$ is in exactly one subset. \\
\\
Fix an equivalence relation $R$ on $A$. \\
For an element $a\in A$, let $[a]$ denote the set $\{b\in A \given
a\sim_R b\}$. \\
We call this set the {\em equivalence class of $a$ under $R$}. \\
We claim that the sets $[a]$ for $a\in A$ constitute a partition of
$A$.

\begin{lemma}
For every $a,b\in A$, either $[a]=[b]$ or $[a]\cap [b]=\emptyset$.
\end{lemma}

\begin{proof}
Fix $a,b\in A$. \\
If $[a]=[b]$ then we are done, so suppose not. \\
Let $c$ be any element that is in one of the sets but not the other. \\
Without loss of generality we can assume that $c \in [b]-[a]$
(we know that either $c\in [b]-[a]$ or $c\in [a]-[b]$.
In the latter case we can simply swap $a$ and $b$ and reduce to 
the first case.).
\\
Similarly, if $[a] \cap [b]=\emptyset$ then we are done, so suppose
not. \\
Let $d$ be any element in $d \in [a] \cap [b]$. \\
\\
We will get a contradiction by showing that $a\sim_R c$ and therefore
that $c\in [a]$. \\
\\
First, $a\sim_R d$ because $d\in [a]$ (note that $d=a$ is a possibility but
this is ok because $R$ is reflexive). \\
Second, $d\sim_R b$ and $b\sim_R c$ because both $c,d\in [b]$ and $R$
is symmetric. \\
This implies, by transitivity, that $d\sim_Rc$. \\
Finally, by transitivity, $a\sim_R c$ because $a\sim_R d$ and $d\sim_R c$.
\end{proof}

Conversely, any partition $\{A_1,\cdots,A_k\}$ of $A$ defines an
equivalence relation by letting $a\sim_R b$ iff $a$ and $b$ are in the
same $A_i$. \\
(Note ``same''.) \\
This is an equivalence relation:\\
\begin{itemize}
\item Reflexivity: for all $a$ we know $a\in A_i$ for some $i$,
by definition of partition. Clearly $a$ and $a$ are in the same $A_i$,
and $a\sim_R b$.
\item
Symmetry: Assume $a\sim_R b$, that is $a$ and $b$ are in the 
same $A_i$. Also $b$ and $a$ are in the same $A_i$
and therefore $b\sim_R a$.
\item
Transitivity: Assume $a\sim_R b$ and $b\sim_R c$.
By definition of $\sim_R$, $a$ and $b$ are in the same 
$A_i$ for some $i$, and $b$ and $c$ are in the same $A_j$
for some $j$.
But by definition of partition $b$ cannot be in two different $A_i$'s.
So, it must be $A_i=A_j$ and $a$ and $c$ are in the same 
$A_i$, proving $a\sim_R c$.
\end{itemize}

Therefore, we can look at partitions and at equivalence relations as
the same thing. 

\subsection{Integers modulo $m$}

In Lecture 4, we considered the modulo operation, for a particular
positive integer $m$, as a function. \\
Another way to think of it is as a definition of an equivalence
relation on integers (positive, negative and $0$):

\begin{defin}
If $a$ and $b$ are integers, then we say $a=b \pmod m$ if $m \mid
(a-b)$.  
\end{defin}

An equivalent formulation says that $a=b+km$ for some integer $k$.  

\begin{theorem}
Equality modulo $m$ is an equivalence relation.
\end{theorem}

\begin{proof}
We need to show the relation is reflexive, symmetric, and transitive.
\begin{itemize}
\item {Reflexive:} Clearly $m \mid a-a = 0$, so $a=a \pmod m$.
\item {Symmetric:} If $a=b+km$ then $b=a+(-k)m$.
\item {Transitive:} If $a=b \pmod m$ and $b = c \pmod m$, this means that
  $m \mid (a-b)$ and $m \mid (b-c)$.  It follows from a theorem in the
  previous lecture that $m \mid (a-b)+(b-c) = a-c$.
\end{itemize}
\end{proof}

The equivalence class of $a$ is simply the set 
$[a] = \{ b \in {\Bbb Z} \mid a = b \pmod m \}$, or
$\{ km + a \mid k \in {\Bbb Z}\}$. \\
\\
It turns out that we can extend a lot of standard arithmetic to work
modulo $m$. \\
In fact, we can define notions of sum and product for the equivalence
classes mod $m$. \\
For example, we define $[a]+[b]$, given two equivalence classes mod
$m$, to be the equivalence class $[a+b]$. \\
This is not as obvious as it seems: notice
that the result is given in terms of $a$ and $b$, two selected
representative from the equivalence classes, but that it is supposed
to apply to the equivalence classes themselves.  \\
To prove that this works, we have to show that it doesn't matter which
representatives of the equivalence classes we choose:

\begin{lemma}
If $a=x \pmod m$ and $b=y \pmod m$ then $(a+b) = (x+y) \pmod m$.  
\end{lemma}

\begin{proof}
$m \mid (a-x)$ and $m \mid (b-y)$ so $m \mid ((a-x)+(b-y)) =
(a+b)-(x+y)$,  
\end{proof}

It follows that if we are interested only the result of the addition
modulo $m$, which is the case, for example, in the RSA cryptosystem,
that we can at any time replace a given number with a different number
equivalent to it, without changing the value (equivalence class) of
the final answer. \\
\\
The same fact can be proven for multiplication.  \\
This is the basis of the fast exponentiation algorithm presented in
the next subsection.

\subsection{Fast Exponentiation}

Let's say that we want to compute $a^n \bmod b$.  \\
Here's the crude, obvious way to do it:

\begin{enumerate}
\item Compute $a^n$.
\item Divide by $b$ and take the remainder.
\end{enumerate}

There are actually two big problems with this. \\
First of all, $a^n$ could be a gigantic number. \\
And secondly, we're using $n$ multiplications,
which could take a long time if $n$ is large.
\\
Let's correct the first problem first.

\begin{enumerate}
\item Initialize $p = 1$.
\item Repeat $n$ times:
\begin{itemize}
\item[(i)] Set $p \leftarrow ap$.
\item[(ii)] Set $p \leftarrow p \bmod b$.
\end{itemize}
\item Return $p$.
\end{enumerate}

That is, we are reducing mod $b$ after every multiplication. \\
To see why this works, we have to use the fact that
$(x\cdot y) \bmod b = ((x \bmod b) \cdot y) \bmod b$. \\
Let's prove the correctness of the algorithm more formally.

Notice that we are using the expression $x \bmod b$ to 
denote the result of computing the remainder of $x$ 
when divided by $b$. In particular $x\bmod b$ is a number 
between $0$ and $b-1$.

Clearly the algorithm always terminates as it executes the loop 
only a finite number of times, namely $n$.
We prove by induction on the number of iterations that 
at the beginning of the $i^{th}$ iteration
we have $p=a^i \bmod b$.\\
BASE CASE: At iteration $0$ we have $p = 1 = a^0 \bmod b$.\\
INDUCTIVE STEP: Assume that at iteration $i$ we have 
$p= a^i \bmod b$. Executing the loop one more time 
we get $p'\leftarrow (ap) \bmod b$.
Using the induction hypothesis this is equal to 
$p' = (a\cdot (a^i \bmod b)) \bmod b = (a^{i+1})\bmod b$.\\
After executing the loop $n$ times we will have $p = a^n\bmod b$
and the algorithm terminates with the right answer.
\\

Now for the second problem: \\
To reduce the number of multiplications, we try to use 
squaring whenever possible.

Algorithm {\tt fast-exp(a,n,b)}
\begin{enumerate}
\item If $n = 1$ then return $a \bmod b$.
\item Else if $n$ is odd return $a *$ {\tt fast-exp(a,n-1,b)} $\bmod b$.
\item Else ($n$ is even), set $p \leftarrow$ {\tt fast-exp(a,n/2,b)} and
return $p^2 \bmod b$.
\end{enumerate}

We will prove three properties about the {\tt fast-exp} algorithm.
\begin{enumerate}
\item Termination: for any $a,b,n$ algorithm 
  {\tt fast-exp(a,n,b)} terminates.
\item Correctness: when {\tt fast-exp(a,n,b)} terminates
  it returns the right answer, that is 
  {\tt fast-exp(a,n,b)}$ = a^n \bmod b$.
\item Efficiency: for any $a,b,n$ algorithm 
  {\tt fast-exp(a,n,b)} terminates after at most $2\log_2 n$
  recursive calls.
\end{enumerate}

We will prove all three properties by strong induction on 
the value of $n$ passed to the algorithm.\\
BASE CASE: If $n=1$ the algorithm terminates immediately
 with result $a = a^1 \bmod b$.
 The number of recursive calls is $0 = 2\log_2 1$.\\
INDUCTIVE STEP: Assume that properties 1,2 and 3 are 
satisfied whenever the algorithm is called with values 
of $n$ smaller then some $m>1$.
We will prove that {\tt fast-exp(a,m,b)} terminates
with result $a^m\bmod b$ after at most
$2\log_2 m$ recursive calls.
Consider the execution of {\tt fast-exp(a,m,b)}.
The test at step 1 fails because $m>1$ by assumption. 
We distinguish two cases:
\begin{itemize}
 \item If $m$ is even, the test at step 2 fails, 
  the test at step 3 succeeds and 
  {\tt fast-exp(a,m/2,b)} is called recursively. 
  Since $m/2<m$, we know this procedure call terminates
  with result $p\leftarrow a^{m/2}\bmod b$.
  Therefore,  {\tt fast-exp(a,m,b)} terminates
  with result $p^2\bmod b = (a^{m/2}\bmod b)^2 \bmod b = a^m\bmod b$.
  The number of recursive calls is $1$ plus the 
  number of recursive calls made by {\tt fast-exp(a,m/2,b)}.
  This is by induction at most $2\log_2 (m/2) = 2\log_2m-2$,
  so the total number of recursive calls is at most 
  $1+ 2\log_2m-2<2\log_2 m$.
 \item If $m$ is odd, the test  at step 2 succeeds and 
  {\tt fast-exp(a,m-1,b)} is called recursively. 
  Since $m-1<m$, we know this procedure call terminates
  with result $a^{m-1}\bmod b$.
  Therefore, also {\tt fast-exp(a,m,b)} terminates
  with result $a\cdot (a^{m-1}\bmod b) \bmod b = a^m\bmod b$.
  To bound the number of recursive calls we need to examine 
  the execution of the algorithm more in depth.
  The number of recursive calls is $1$ plus the 
  number of recursive calls made by {\tt fast-exp(a,m-1,b)}.
  Now, if $m$ is odd then $m-1$ is even and 
  {\tt fast-exp(a,m-1,b)} will call recursively 
  {\tt fast-exp(a,(m-1)/2,b)} which by inductive hypothesis will 
  make at most $2\log_2((m-1)/2)$ recursive calls.
  So, the total number of recursive calls made by 
  {\tt fast-exp(a,m-1,b)} is at most 
  $2+2\log_2((m-1)/2)=2\log_2(m-1) < 2\log_2 m$.
\end{itemize}
This concludes the proof of correctness of the algorithm.

Since the algorithm executes only one multiplication for 
each recursive call, it uses at most $2 \log n$ multiplications 
to compute $a^n \bmod b$
(a huge savings over $n$ multiplications even for moderate values of $n$).




\section{Partial Orders}

We are covering a series of mathematical structures: \\
natural numbers and integers, \\
sets, \\
relations (including equivalence relations), \\
functions. \\
\\
Next kind of structure: partial order. \\
A particular type of binary relation. \\
\\
Important in computer science. \\
Applications to task scheduling, \\
Database concurrency control, \\
Logical time in distributed computing \\

\subsection{Definitions}

\subsubsection{Partial Order Definition}

\begin{defin} 
A binary relation $R \subseteq A \times A$ is a {\bf partial order\/}
if it is reflexive, transitive, and antisymmetric.
\end{defin}

Quick review:
\begin{itemize}
\item reflexive: $a R a$
\item transitive: $a R b \wedge b R c \Rightarrow a R c$
\item symmetric: $a R b \Rightarrow b R a$
\item antisymmetric: $a R b \wedge b R a \Rightarrow a=b$.
  i.e., $\forall a \neq b, a R b \Rightarrow \neg b R a$. Means NEVER
  symmetric!
\end{itemize}

For comparison: \\
Recall that a relation that is reflexive, transitive, and symmetric
is an equivalence relation. \\
\\


The reflexivity, antisymmetry and transitivity properties are abstract
properties that generally describe ``ordering'' relationships. \\ 

So we often write an ordering-style symbol like $\preceq$ instead of
just a letter like $R$, for a p.o. relation.  This lets us use
notation similar to for $\le$.  For example, we write $a \prec b$ if
$a \preceq b$ and $a \ne b$.  Similarly, we write $b \succeq a$ as
equivalent to $a \preceq b$.

But this could be misleading -- note that $\geq$ is a p.o. on natural
numbers, as well as $\leq$.  If we use the $\preceq$ symbol for $\geq$,
things look really funny. 
In cases like this, better to use $R$.
\\
A partial order is always defined on some set $A$.  The set together
with the partial order is called a ``poset'':
\begin{defin}
A set $A$ together with a partial order $\preceq$ is called a 
{\bf poset\/}  $(A, \preceq)$.
\end{defin}



Examples of posets:
\begin{itemize}
\item $A=\naturals, R=\leq$, easy to check reflexive, transitive, antisymmetric
\item $A=\naturals, R=\geq$, same.
\item $A=\naturals, R=<$, NOT because not reflexive
\item $A=\naturals, R=|$ (divides), easy to check reflexive,
  transitive, antisymmetric
\item $A=P(\naturals), R=\subseteq$, check reflexive: $S \subseteq S$,
  transitive: $S \subseteq S' \wedge S' \subseteq S'' \Rightarrow S
  \subseteq S''$, antisymmetric: $S \subseteq S' \wedge S' \subseteq S
  \Rightarrow S = S'$.
\item $A=$ ``set of all computers in the world'', $R=$ ``is (directly or
indirectly) connected to''. \\
NOT a p.o. because it is not true that $a R b \wedge b R a \Rightarrow
a=b$. \\
In fact, it is symmetric, transitive.  Equivalence relation.
\item $A=$ ``set of all propositions'', $R=\Rightarrow$, NOT because
  not antisymmetric. Not symmetric either, so not equivalence
  relation.
\item $A=$ some set of tasks, $R=$ ``must be done before'', according
to some task specification. \\ 
Antisymmetry: Does $a R b \wedge b R a \Rightarrow a=b$? \\
Hopefully, because otherwise $a, b$ can never be done! 
So we will only deal with task specifications with this property. \\
Transitive. \\
BUT not reflexive. If we want to treat this as a partial order, we
must make a small modification of $R$ to
``either = or must be done before''.
\end{itemize}

\subsubsection{Reflexive and Transitive Closures}

The construction we just did is called ``taking the reflexive
closure''. \\
In general, if we have a binary relation $R$ that is antisymmetric and
transitive (but not necessarily reflexive), we can turn it into a 
partial order simply by adding in all pairs of the form $(a,a)$. \\
Adding in these pairs does not mess up the antisymmetry or
transitivity properties. \\
(Check this.) \\
\\
A related construction is ``taking the transitive closure''. \\
It starts with a relation $R$ that is antisymmetric and reflexive, but
not necessarily transitive. \\
However, it has no ``cycles'', i.e., no instances where
$(a_1,a_2) \in R$, \\
$(a_2,a_3) \in R$, \\
...
$(a_{k-1},a_k) \in R$, and
$(a_k,a_1) \in R$, \\
where each pair consists of distinct elements, and where $k \geq 2$. \\
\\
E.g., ``$S$ is a subset of $S'$, with either the same number of elements
(so $S = S'$) or else exactly one fewer''. \\
This is reflexive, antisymmetric, not transitive. \\
But no cycles. \\
\\
Then add in all the transitively-determined pairs. \\
What are these? \\
Well, if $(a,b)$ and $(b,c)$ are in $R$, then must add in $(a,c)$.  \\
But not enough just to do this for the original pairs.  \\
Must repeat, e.g., if $(c,d) \in R$, must now add in $(a,d)$, etc. \\
In general, must add in any pair $(a_1,a_k)$ such that 
all the pairs $(a_1,a_2)$, $(a_2,a_3)$, ..., $(a_{k-1},a_k)$ are in
$R$. \\
Claim:  this is enough to make the result transitive. \\
(For any two pairs assembled in this way that match up, the transitive
combination is also included.) \\
Moreover, adding all these pairs doesn't mess up reflexivity
(obviously) or antisymmetry (because the original relation had no
cycles). \\
So the result is a partial order. \\
\\
E.g., in the subset example above, transitive closure adds in all
subset pairs without any restriction on the number of elements in the
set difference. \\
\\
Could also start with not-necessarily reflexive, but antisymmetric and
cycle-free, and take the ``reflexive transitive closure'' to turn it
into a p.o.  \\
Just do both of the above constructions. \\
\\
E.g., ``$S$ is a subset of $S'$, with $S$ missing exactly one element from
$S'$.''

\subsubsection{No Cycles}

In taking the transitive closure, we had to start with a relation
without cycles.  \\
In fact, it turns out that {\em every\/} partial order is cycle-free:

\begin{theorem}
A partial order has no cycles of length $\geq 2$, i.e., no sequences
of pairs of the form
$(a_1,a_2)$, $(a_2,a_3)$, ...$(a_{k-1},a_k)$, and $(a_k,a_1)$,
$k \geq 2$, all in $R$, each pair consisting of distinct elements. 
\end{theorem}

\begin{proof}
By contradiction. \\
Assume that such a cycle exists in $R$. 
Then by transitivity (maybe applied several times), $(a_1,a_k) \in R$.
Since also $(a_k,a_1) \in R$,
antisymmetry implies that $a_1 = a_k$, which is a contradiction.
\end{proof}

\subsubsection{Posets and Hasse Diagrams}

Representing posets: \\
\\
Can usually be nicely drawn as a graph (Informally, dots and arrows
.) \\
Draw an arrow from $a$ to $b$ if $a R b$. 
(Similar to what we've already done with relations.) \\
\\
E.g.: Draw a poset that describes the order in which one
would put on clothes. \\
The set is of clothes, and the partial order
specifies what should be put on before what.

\setlength{\unitlength}{0.012500in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(220,75)(20,760)
\thinlines
\put(165,817){\vector( 0,-1){ 20}}
\put(240,817){\vector( 0,-1){ 20}}
\put(240,787){\vector( 0,-1){ 20}}
\put(165,787){\vector( 0,-1){ 20}}
\put( 25,797){\vector( 0,-1){ 25}}
\put( 85,797){\vector( 0,-1){ 25}}
\put(180,787){\vector( 3,-1){ 51}}
\put(178,765){\vector( 1, 0){ 53}}
\put(153,795){\vector(-4,-1){88}}
\put(155,790){\makebox(0.1111,0.7778){\SetFigFont{5}{6}{rm}.}}
\put(153,790){\vector(-2,-1){ 32}}
\put( 20,800){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}left sock}}}
\put( 20,765){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}left shoe}}}
\put( 80,800){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}right sock}}}
\put(235,820){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}shirt}}}
\put(155,820){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}underwear}}}
\put(235,790){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}sweater}}}
\put(235,760){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}jacket}}}
\put(155,760){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}belt}}}
\put(155,790){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}pants}}}
\put( 80,765){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}right shoe}}}
\end{picture}

Notice that some arrows are left out.
E.g., underwear precedes belt. \\
But all those omitted follow from transitivity. \\
Loops (which express reflexive relationships) also aren't shown. \\
\\
\begin{defin}
A {\bf Hasse diagram\/} for a poset $(A,\preceq)$ is a graph: \\
- whose vertices are the elements of $A$, \\
- whose edges (arrows) represent pairs $(a,b)$, where $a \leq b$ but
$a \neq b$, \\
- that contains no ``transitive edges'', \\
- for which all pairs in $\preceq$ are obtainable by transitivity from
edges in the diagram.
\end{defin}

To convert the diagram above to an ``official form'' Hasse diagram,
must remove the pants $\rightarrow$ jacket arrow. \\
\\
Notice that the graph above has directed paths, that is,
sequences of nodes $a_1 \cdots a_n$ where $(\forall i<n) a_i R a_{i+1}$. \\
However, it does not have any directed cycles, a directed path where
$a_1 = a_n$. Good thing, because otherwise we could get stuck trying
to get dressed. \\
As proved above, this is true of all posets. 

\subsubsection{Partial vs. Total Orders}

A partial order is called {\em partial \/} because it is not necessary
that an ordering exist between every pair of elements in the set. \\
\\
E.g., Lshoe and Rshoe have no prescribed ordering between them. \\
\\
E.g., For two sets, it's not necessary that either be a subset of the
other. \\
\\
When there is no prescribed order between two elements we say that they
are ``incomparable''. 

\begin{defin}
$a$ and $b$ are {\bf incomparable\/} if neither $a \preceq b$ nor $b
\preceq a$.
\end{defin}

\begin{defin}
$a$ and $b$ are {\bf comparable\/} if $a \preceq b$ or $b \preceq a$.
\end{defin}

E.g., for subsets of $\naturals$, $\{ 1,2,3 \}$ and $\{ 2,3,4 \}$ are
incomparable. \\
\\
However, a partial order need not have incomparable elements. \\
As a special case, we can have a p.o. in which there is a specified
order between every pair of elements. \\
Such partial orders are called ``total orders''.

\begin{defin}
A poset $(S, \preceq)$ is {\bf totally ordered\/} if 
$(\forall a,b \in S) [ a \preceq b \vee b \preceq a ]$.
\end{defin}

Examples:
\begin{itemize}
\item $S=\naturals, \preceq = \leq$
\item $S=\naturals, \preceq = |$, NOT because neither $3|5$ nor $5|3$
\item $S=P(\naturals), \preceq = \subseteq,$ NOT because neither $\{3\}
  \subseteq \{5\}$ nor $\{5\} \subseteq \{3\}$ 
\item Lexicographic order on pairs of natural numbers, defined by:
$(a_1,a_2) \preceq (b_1,b_2)$ if and only if either $a_1 < b_1$, 
or else $a_1 = b_1$ and $a_2 \leq b_2$. \\
Do some examples here. $(2,37) \preceq (2,38) \preceq (3,2)$. \\
``Lexicographic'' because like dictionary.
\end{itemize}

The Hasse diagram of a total order looks like a line. 

This has been the basic material. \\
Book has a bit more: \\
minimal, maximal elements (nothing smaller, nothing larger) \\
lower bounds, upper bounds (for a subset -- $\leq$ everything in the
subset, or $\geq$) \\
lub, glb, \\
lattices. \\
Read these parts, do some for HW. \\
\\

\subsection{Topological Sorting}

Sometimes when we have a partial order, \\
e.g., of tasks to be performed, \\
we want to obtain a consistent total order, \\
e.g., an order in which to perform all the tasks, one at a time, so as
not to conflict with the precedence requirements. \\
\\
The task of finding an ordering that is consistent with a partial
order is known as ``topological sorting''.
I think because the sort is based only on the shape (topology) of the
poset, and not on the actual values.

\begin{defin}
  A {\bf topological sort\/} of a finite poset $(A, \preceq)$ is a
  total ordering of all the elements of $A$, $a_1, a_2, \cdots, a_n$
  \\ in such a way that $\forall i < j$, either $a_i \prec a_j$ or
  $a_i$ and $a_j$ are incomparable in $(A, \preceq)$ (equivalently, it
  is not the case that $a_j \prec a_i$).
\end{defin}

For example, underwear, shirt, Lsock, Rsock, pants, sweater, Lshoe, Rshoe,
belt, jacket is a topological sort of how to dress. \\
\\
One of the nice facts about posets is that such an ordering always
exists and is even easy to find:

\begin{theorem}
Every finite poset has a topological sort.
\end{theorem}

We'll prove this by induction. \\
On the number of elements in the domain of the poset. \\
The basic idea is to pick a ``smallest'' element to start and then
proceed recursively.

\begin{defin}
A {\bf minimal\/} element $a$ in a poset $(A, \preceq)$ is one for
which $(\forall b \in A) [ a \preceq b \vee a \mbox{ and } b \mbox{ are
incomparable }]$.  Equivalently, it is an element $a$ for which no $b
\prec a$.
\end{defin}

Notice that there can be more than one minimal element. There are 4 in
the clothes example: Lsock, Rsock, underwear, shirt.

\begin{lemma}
Every finite poset $(A,\preceq)$ has a minimal element.
\end{lemma}

\begin{proof}
By contradiction. \\
Assume no minimal elements.
Then $(\forall a \in A) (\exists b) [ b \prec a ]$.  (Recall $b \prec
a$ means $b \preceq a$ and $b \ne a$.\\
\\
Start from any element of $a$ and construct a sequence of successively
smaller elements, by using the fact above. \\
At each step, can get another element preceding the most recent. \\
Can't repeat any of the previous ones in the chain, since that would
create a cycle (and we know a poset has no cycles). \\
So just keep going forever, creating an infinite chain of distinct
elements.  \\
\\
But this is nonsense, because $A$ is a finite set. \\
Contradiction, so there must be some minimal element.

More formally, based on assumption, can give a recursive definition
for a sequence $a_k$.  Set $a_1$ to be some element of $A$, and define
$a_{k+1}$ to be some element $\prec a_k$ (such an element always
exists by assumption).  But now suppose $A$ has $n$ elements.  Then
consider the values $a_1,\ldots,a_{n+1}$ of our sequence.  By
pigeonholing (number $1$ to $n=1$ are pigeons, values in $A$ are
holes) there must be $j \ne k$ with $a_j=a_k$.  But this means we have
a cycle on $a_j \ldots a_k$.  We've already argued this can't happen
in a poset unless all of $a_j,a_{j=1},\ldots,a_k$ are equal.  But this
violates the requirements that $a_j \ne a_{j=1}$.  So we have a
contradiction.  

A different way to prove this is by induction.  You'll do this for
homework (actually, the pigeonhole argument is also using induction;
it's hiding in the proof of the pigeonhole principle).
\end{proof}

Note that an infinite poset might have no minimal element.  Consider
$\integers$.  


Now we can prove the existence of the topological sort.

\begin{proof}
By (ordinary) induction. \\
$P(n)$ = any poset with $n$ elements has a topological sort. 
\\
Base case $(P(1))$: for a poset with one element the one element makes a
topological sort. \\
\\
Inductive step: Assume $(P(n))$. Consider a poset of $n+1$ elements. \\
By lemma it has a minimal element $u$. \\
Consider the same relation on $A-\{u\}$. \\
It is easy to check that it is still reflexive, transitive, and
anti-symmetric. \\
So by the inductive hypothesis, $A-\{u\}$ has a topological sort. \\
We put $u$ first and now have a topological sort on the whole thing. \\
(According to def of topological sort -- $u$ can't be in
the wrong order w.r.t. any of the later elements.) \\
\\
Thus every poset has a topological sort, by induction.
\end{proof}

E.g.: Dressing example: \\
Construct ordering by picking one at a time. \\
In each case, look at the remaining p.o. for remaining elements. \\
Lsock, shirt, sweater, Rsock, underwear, pants, Lshoe, belt, jacket, Rshoe

E.g., subsets of $\{ 1,2,3,4 \}$ \\
$\emptyset$ is the unique minimal element, then have choices, e.g., do: \\
$\{ 1 \}$, $\{ 2 \}$, $\{ 1,2 \}$ \\
$\{ 3 \}$, $\{ 1,3 \}$, $\{ 2,3 \}$, $\{ 1,2,3 \}$, \\
$\{ 4 \}$, $\{ 1,4 \}$, $\{ 2,4 \}$, $\{ 3,4 \}$, \\
$\{ 1,2,4 \}$, $\{ 1,3,4 \}$, $\{ 2,3,4 \}$, $\{ 1,2,3,4 \}$

\subsection{Parallel Task Scheduling}

When elements of a poset are tasks that need to be done and the partial
order is precedence constraints, topological sorting provides us with
a legal way to execute tasks sequentially, i.e., without violating any
precedence constraints. \\
But what if we have the ability to execute more than one task at the
same time? \\
For example, say tasks are programs, partial order indicates data
dependence, and we have a parallel machine with lots of processors
instead of a sequential machine with only one. \\
How should we schedule the tasks? \\
Our goal should be to minimize the total {\em time\/} to complete all
the tasks. \\
(For simplicity, say all the tasks take the same amount of time and
all the processors are identical.)
\\
So: \\
Given a finite poset of tasks, how long does it take to do them all,
in an optimal parallel schedule? \\
Assume that all tasks take 1 unit of time and we have an unlimited number
of processors. \\
\\
Use partial order concepts to analyze this problem. \\
\\
On clothes example, we could do all the minimal elements first (Lsock,
Rsock, underwear, shirt), remove them and repeat. \\
Need lots of hands, or maybe dressing servants. \\
(So do pants and sweater next, and then Lshoe, Rshoe, and belt, and
finally jacket.) \\
\\
Can't do any better, because the sequence shirt, pants, belt, jacket
must be done in that order. \\
A sequence like this is called a chain.

\begin{defin}
A {\bf chain\/} in a poset is a sequence of elements of the domain,
each of which is smaller than the next in the partial order 
($\preceq$ and $\neq$). 
\end{defin}

Clearly, parallel time $\geq$ length of any chain. \\
This is true, because if we used less time, then by pigeonhole two
tasks in the chain would have to be done at the same time, but by
definition of chain this violates precedence constraints. \\
\\
A longest chain is also known as a {\em critical path}. \\
\\
So we need at least $t$ steps, where $t$ is the length of the longest
chain. \\
Fortunately, it is always possible to use only $t$:

\begin{theorem}
Given any finite poset $(A, \preceq)$ for which the longest chain
has length $t$, it is possible to partition $A$ into $t$ subsets,
$A_1, A_2, \cdots, A_t$ such that 
$(\forall i \in [1,t]) (\forall a \in A_i) 
(\forall b \preceq a, b \neq a) 
[ b \in A_1 \cup A_2 \cup \cdots \cup A_{i-1} ]$.
\end{theorem}

That is, can divide up the tasks into $t$ ``groups'' so that for
each group $i$, all tasks that have to precede tasks in $i$ are in
smaller-numbered groups. 

\begin{corollary}
For $(A, \preceq)$ and $t$ as above, it is possible to schedule all
tasks in $t$ steps.
\end{corollary}

\begin{proof}
$\forall i$, schedule all elements of $A_i$ at time $i$. \\
This satisfies the precedence requirements, because all tasks that
must precede a task are scheduled at preceding times. \\
\end{proof}

\begin{corollary}
parallel time = length of longest chain
\end{corollary}

So it remains to prove the partition theorem:

\begin{proof}
Use the rule: \\
Put each $a$ in set $A_i$, where $i$ is the length of the longest chain
ending at $a$. \\
\\
See this works: \\
It gives just $t$ sets, because the longest path is length $t$. \\
\\
Need to show $(\forall i) ( \forall a \in A_i)$ \\
$(\forall b \preceq a, b \neq q) 
[ b \in A_1 \cup A_2 \cup \cdots \cup A_{i-1} ]$. \\
\\
Proof by contradiction: \\
If $b \not\in A_1 \cup A_2 \cup \cdots \cup A_{i-1}$ then there is a path
of length $>i-1$ ending in $b$, \\
which means there is a path of length $>i$ ending in $a$, \\
which means $a \not \in A_i$. \\
This is a contradiction.
\end{proof}

So with an unlimited number of processors, the time to complete all
the tasks is the length of the longest chain.  \\
\\
It turns out that this theorem is good for more than parallel
scheduling. It is usually stated as follows.

\begin{corollary}
If $t$ is the length of the longest chain in a poset $(A, \preceq)$
then $A$ can be partitioned into $t$ antichains.
\end{corollary}

\begin{defin}
An {\bf antichain\/} is a set of incomparable elements.
\end{defin}

This is a corollary because we showed before how to partition the tasks
into sets that could be done at the same time, which means that these
were incomparable. (If any two comparable then one would have to
precede other and so couldn't be done at same time.)

Review:
\begin{itemize}
\item {\bf chain\/}: all elements comparable (total order)
\item {\bf antichain\/}: all elements incomparable
\end{itemize}

\subsection{Dilworth's Theorem}

We can use this to prove a famous result about posets:

\subsubsection{The Basics}

\begin{theorem}
(Dilworth) $\forall t$, every poset with $n$ elements must have a
chain of size $ > t$ or an antichain of size $\geq \frac{n}{t}$.
\end{theorem}

Ex: Dressing poset.  $n = 10$.
Try $t = 3$.  Has chain of length $4$. \\
Try $t = 4$.  Has no chain of length $5$, but has antichain of size 
$4 \geq \frac{10}{4}$. \\

\begin{proof}
Assume there is no chain of length $> t$. \\
So, longest chain has length $\leq t$. \\
Then by previous result, the $n$ elements can be partitioned into
$\leq t$ antichains. \\
(Can assume exactly $t$, since could have some empty.) \\
So there is an antichain with $\geq \frac{n}{t}$
elements. \\
As needed.
\end{proof}

For instance: 

\begin{corollary}
Every poset with $n$ elements has a chain of length $> \sqrt{n}$
or an antichain of size $\geq \sqrt{n}$.
\end{corollary}

Set $t= \sqrt{n}$ and this is immediate.

Posets arise in all sorts of contexts, and when they do Dilworth's
theorem can have interesting implications. 

\subsubsection{Increasing and Decreasing Sequences}

A mathematical example: \\
In any sequence of $n$ different numbers, there is either an
increasing subsequence of length $> \sqrt{n}$ or
a decreasing subsequence of length $\geq \sqrt{n}$. \\
\\
E.g., 6 4 7 9 1 2 5 3 8 has the decreasing sequence 6 4 1 and
the increasing sequence \\ 1 2 3 8.
That is, it has both. \\
\\
We can get this using Dilworth's theorem; the trick is to define the 
appropriate poset. \\
\\
The domain is the set of values in the sequence. \\
\\
For the ordering, define $a \preceq b$ if either $a = b$, or else
($a < b$ and $a$ comes before $b$ in the sequence). \\
\\
Check this is reflexive (stated explicitly), 
transitive, and antisymmetric. \\
\\
A chain corresponds to a sequence that increases in value and moves to
the right, that is, an increasing sequence. \\
\\
But what does an antichain correspond to?

\begin{lemma}
If $a,b$ are incomparable and $a > b$ then $a$ precedes $b$ in the
sequence. 
\end{lemma}

\begin{proof}
By contradiction. \\
Assume $b$ precedes $a$. \\
Then $b < a$ and $b$ precedes $a$, so $b \preceq a$ in the partial
order. \\
This contradicts our assumption that $a$ and $b$ are incomparable.
\end{proof}

Extend this to more than 2 elements:

\begin{lemma}
If $a_1, a_2, \cdots , a_t$ are incomparable and $a_1 > a_2 > \cdots
  > a_t$ then $a_1, a_2, \cdots a_t$ form a decreasing subsequence.
\end{lemma}

\begin{proof}
$(\forall i)$,
the fact that $a_i > a_{i+1}$ implies that
$a_i$ precedes $a_{i+1}$ in the sequence, by the previous lemma.
\end{proof}

So given an antichain, arrange the elements so that they are in
decreasing order and the ``left of'' relation follows, giving a
decreasing subsequence.

Dilworth's theorem implies that there is either a chain of size 
$> \sqrt{n}$ or an antichain of size $\geq \sqrt{n}$. \\
By the analysis just above,
this yields either an increasing sequence of length $> \sqrt{n}$
or a decreasing subsequence of length $\geq \sqrt{n}$. 

\subsubsection{Number of Comparisons for Database Setup and Searching}

Consider a ``database'' of records with (distinct) numerical keys. \\
Suppose when records are inserted, some comparisons of these keys
occur, and the database maintains a record of these comparisons. \\
\\
E.g., at one extreme, database could be kept totally sorted. \\
E.g., totally unsorted. \\
\\
Each comparison costs something -- the time to perform it. \\
\\
Now suppose that after this setup, a search is to be performed on the
database to determine if a record with a particular key $k$ appears
anywhere in the database.  \\
The search is supposed to be based on comparisons only. \\
Then the worst-case number of comparisons it takes depends on the
database partial order. \\
\\
E.g., if db is totally sorted, can compare with middle elements, then
middle of halves, etc.  \\
Binary search.  \\
Approximately log n comparisons needed. \\
\\
E.g., if db totally unsorted, then in the worst case, have to compare
with every element.  \\
This is because each comparison with an element in the db gives no
information about whether the sought element might be any of the
others in the db. \\
\\
In fact, if there is an antichain of size $k$, then in the worst case,
the new key could be any of the elements of the antichain. \\
So in the worse case, have to compare with all $k$, since each one
gives no information about the others. \\
\\
See what Dilworth's theorem says about the database partial order. \\
For any $t$, either have a chain of length $> t$, or an antichain of
size $\geq \frac{n}{t}$.  \\
Setting up a chain of length $> t$ requires at least $t$ comparisons. \\
The antichain requires at least $\frac{n}{t}$ comparisons. \\
So this says that we have to pay the cost sometime -- tradeoff --
either we do $> t$ comparisons while setting up the db, or at
least $\frac{n}{t}$ in the worst case to search the db. \\
\\
More refined analysis is possible. \\
That's because reliably setting up a long chain, say length $t$, in
the database actually requires {\em more} than $t$ comparisons.  \\
Because in the worst case, the comparisons won't come out in the most
desirable way, setting up a linear chain. \\
Ex: Compare $a$ and $b$, find out $a > b$.  Then compare $b$ and $c$.
Would be nice if we found out $b > c$.  But we might not -- might get
$c > b$.  Then we don't have a chain of length $3$, but still an
antichain of size $2$.  To get a chain, need an additional comparison. \\
\\
Return to this in analysis of algorithms.

\subsection{Logical Time in Distributed Systems}

One last use of p.o.  \\
No use of powerful theorems here -- just showing an important place
where a p.o. is used as a mathematical model for a real computational
situation.  \\
\\
A distributed computation consists of a sequence of ``steps''
occurring at each of a collection of $n$ ``processes''. \\
In addition to performing local steps, the processes can send messages
to each other. \\
So, have $\ms{send}$ steps and $\ms{receive}$ steps. \\
\\
The processes are mostly working independently. \\
The only interaction is by means of the messages. \\
So, the only dependence among steps (feeding data from one to another,
using results to make decisions, etc.) is by: \\
\begin{enumerate}
\item
Successive steps of same process.
\item
Send vs. receive of same message.
\end{enumerate}
Taking reflexive transitive closure leads to partial order describing
the dependence among the computation steps. \\
\\
Basic theorem:  Any topological sort of this p.o. describes a possible
sequence in which all the distributed steps could occur. \\
(No changes in what happens based on reordering incomparable steps.) \\
\\
Example of what one might do with this p.o.: \\
Assign ``logical time'' to every step. \\
Instead of an actual real time. \\
The time is just any assignment of real numbers to the steps that is
consistent with the p.o.:  when one step precedes another in the p.o.,
it must have a smaller logical time. \\
\\
Behaves like real time:  Formally, can reorder the steps in logical
time order, still consistent with the p.o.\\
Then the system is behaving as if the logical time were actually real
time. \\
\\
Can be maintained by simple distributed algorithm. \\
(Piggybacks time on message, adjusts local times in response to what
is heard in some messages.) \\
\\
Although logical time isn't the same as actual real time, for many
purposes, it can be used as an adequate substitute. \\
Logical time can be used, e.g., to timestamp files to keep updated
copies consistent.

\end{document}

Problems based on this lecture:

1. (Some basic p.o. defs -- minimal and maximal elements, lower and
upper bounds, etc.)
Section 6.6, Problem 20
and/or Section 6.2, Problem 22

2. (Parallel task scheduling)
Recall the theorem from class that says that if a task graph is 
a partial order with maximum chain length $t$, then the
whole set of tasks can be performed by an unlimited collection of
parallel processors in time $t$.
This theorem was only for the case where all tasks take the same
amount of time, say $1$ time unit.
(a) Apply this theorem to produce an optimal schedule for the
following task graph.
[[[give some graph.  Don't need to make up a cute story -- just some
letters will do]]]
(b) Generalize the theorem to the case where some tasks take $1$ time
unit and others take $2$ times units.
[[[Seems this should work by splitting the time-$2$ tasks into two
tasks, using this to come up with a new length measure for chains,
then applying the old theorem. 
Should take some arguing that everything works, but it seems nice.???]]]

3. This doesn't actually use Dilworth's theorem.
But the students will get to think about chains and antichains.

Suppose a database is stored using a partial order on keys.  
Your job is to design a search strategy to determine if a new key $k$
appears in the database.  
The only thing you are allowed to do in the strategy is to compare two
keys (either two within the database, or the new key $k$ vs. a key in
the database).
The result of such a comparison adds extra edges to the partial order.
The results of earlier comparisons can be used to determine which
comparisons are made later.

For each of the partial orders given below, describe a strategy for
searching for an arbitrary key $k$ in the database.
Your strategy should be designed to use the minimum number you can
achieve, of comparisons, in the worst case.
Say how many comparisons it takes in the worst case, and explain why.

(a) Consists of $2$ antichains $A$ and $B$, each with $5$ elements. 
Each of the $5$ elements of $A$ is known to be less than each of the
$5$ elements of $B$.
[[[Can draw diagram]]]

[[[I get 10 comparisons.  Can achieve by direct comparison of new key
with each element.  Doesn't seem we can better than, though -- every
key needs to be compared to $k$ directly or indirectly.  If we assume
$k$ is between all the elements of $A$ and all the elements of $B$,
then it seems we get no help from the existing edges.  Re-think this
-- if it looks too hard, we could default to the simpler case where we
are only allowed to compare $k$ to an element of the db.]]]

(b) Consists of $2$ chains, each of length $5$.  No relationships are
known between elements of the $2$ chains, however.
[[[Can draw diagram]]]

[[[Here I get $6$, by treating the two chains separately.]]]


\end{document}

