\documentstyle[12pt,twoside,amsfonts,amstex]{article}
\input{macros-course-lectures}

\parskip1.5ex
\parindent0pt
\newtheorem{falsethm}{False Theorem}


\begin{document}

\setcounter{section}{7}

\lecture{notes5}{Class Notes for Lecture 5}{February 20, 1997}

\section{Strong Induction}

\subsection{Definition}

Strong induction is a variant of induction.  To prove $(\forall n \in 
\naturals) P(n)$ 
\begin{itemize}
\item prove $P(0)$
\item prove $((\forall k < n)P(k))$ implies $P(n)$.
\end{itemize}

Actually, we don't even need the base case since
the second case, when specialized to $n = 0$, says 
"prove $((\forall k < 0)P(k))$ implies $P(0)$."
But the hypothesis is {\em vacuously true}, since there's nothing $< 0$. 
So this means you're really forced just to prove the conclusion,
$P(0)$, on its own. 

But never mind if this is confusing -- we'll follow the book and check
base cases since they're generally easy to check anyways. 
Another reason to check the base cases is that often the first case or 
first few cases need a different argument for
the inductive step, show explicitly. 

Again, we can modify this to work from any $k$ that isn't $0$. 

This follows from standard (weak) induction on the proposition
$Q(n)=(\forall k < n)P(k)$ (base case at $n=1$).

\subsection{Nim}

The game of Nim is defined as follows:  Some positive number of sticks are 
placed
on the ground.  Two players take turns removing one, two, or three
sticks.  The player to remove the last stick loses.

\begin{theorem}
The first player loses if the number of sticks, $n$, is $4k+1$ for some $k$,
and wins (if he plays optimally) otherwise.
\end{theorem}

We can formalize this by defining a strategy as a rule for how many
sticks to remove when there are $n$ left.  We then show that if $n=4k+1$, 
then player 2 has a strategy that will force a win for him, otherwise, 
player 1 has a strategy that will force a win for him.

\begin{proof}
By strong induction.  Suppose the theorem is true for numbers less than $n$,
show that it is true for $n$.  So we want to show that if $n=4k+1$ then first 
player loses, otherwise if $n = 4k$, $4k + 2$, or $4k + 3$, then the 
first player wins. 
(We'll see later why this exhausts all possible cases for $n$)

There are four cases: 
\begin{itemize}
\item $n=4k+1$: show that the first player loses.  

Base case: $n = 1$. The first player has no choice but to
remove $1$ stick and lose. \\

Case $n\ne 1$, so $n \geq 5$. \\

Consider what the first player might do to win:  he can choose to remove 1, 2 
or 3 sticks. 

If he removes one stick, the remaining number of sticks is $n-1 = 4k$.  \\
By strong induction, the player who plays at this point should win. \\
so the player who played first will lose. \\

Similarly, if the first player removes two sticks, the remainding amount is
$4(k-1)+3$. He, again, loses by the same reasoning.  \\

And again by removing $3$ sticks, he loses.  \\

So however the first player moves, he loses.

\item $n = 4k$: show that the first player can win.

Have first player remove 3 sticks: the second player then sees $4(k-1)+1$ 
sticks, and loses, by the strong inductive hypothesis.

\item 
$n = 4k+2$: show that the first player can win.

Similar to the previous case when the first player removes one stick. 

\item
$n=4k+3$: show that the first player can win.

Again similar to the previous case when the first player removes two sticks. 

\end{itemize}
From the division algorithm that we'll see later, we know these are all
the possible cases.
\end{proof}

\subsection{Structural Induction on a Recursive Definition}

Induction is often used to prove properties for all elements
of a recursively defined set, like well-formed arithmetic formulas or
binary trees.
This is usually formulated so it looks like a different kind of
inductive method: \\
Two steps:
\begin{itemize}
\item Prove for the ``base cases'' of the definition
\item Prove for the result of any combination rule, assuming it can be applied.
%true for parts. TVI
\end{itemize}
This isn't really different, though.  It's just an instance of strong 
induction on the number of formation steps.
The base cases correspond to $0$ applications of combination operations. 

\subsubsection{Example: Well-formed Formulas}

A well-formed formula is:
\begin{itemize}
\item a number
\item $(f+g)$, $(f-g)$, $(f*g)$, or $(f/g)$ where $f$ and $g$ are
  well-formed formulas.
\end{itemize}

\begin{theorem}
Well-formed formulas always have the same number of left and right
parentheses. 
\end{theorem}

\begin{proof}
The proof is by strong induction on $n$, the number of applications of
the ``combination rules.'' \\
Let $P(n)$ be the proposition that a formula formed by $n$
applications of the combination rules has
the same number of  left and right parentheses.\\

\noindent {\em BASE CASE:} $P(0)$: \\
If the formula is just a number then the proposition 
is obviously true since there are $0$ left and right parentheses.
Hence, $P(0)$ is true.\\

\noindent {\em INDUCTIVE STEP:}   
Assume $P(k)$ to be true, for all $k < n$. Consider a 
formula  formed by $n$ applications of the combination rules. 
There are four cases.
\begin{enumerate}
\item The formula is of the form $(f+g)$, where $f$ and $g$ are well-formed.

Since $f$ and $g$ are well-formed and are formed by fewer than $n$
applications of the combination rules, by the strong inductive hypothesis,
they have an equal number of  left and right
parentheses and hence so does  $(f+g)$.
\item The formula is of the form $(f-g)$. This case is similar to the first.
\item The formula is of the form $(f*g)$. This case is similar to the first.
\item The formula is of the form $(f/g)$. This case is similar to the first.
\end{enumerate}
This completes the inductive step and hence $P(n)$ is true for all nonnegative
integers $n$.
\end{proof}


\subsubsection{Example: Binary Trees}

A {\em binary tree} is one of
\begin{itemize}
\item A {\em node} with no children.
\item A node with an edge to a left child that is a binary tree, and
  no right child.
\item A node with an edge to a  right child that is a binary tree, and 
no left child.
\item A node with edges to left and right binary-tree children.
\end{itemize}


\begin{theorem}
The number of edges in any binary tree is exactly one fewer than the
number of nodes. 
\end{theorem}

\begin{proof}
The proof is by strong induction on $n$, the number of applications of
the ``combination rules.'' \\
Let $P(n)$ be the proposition that a binary tree formed by $n$
applications of the combination rules has $1$ fewer edge than nodes.\\

\noindent {\em BASE CASE:} $P(0)$ is true since a binary tree that is a node
with no children has $0$ edges.\\

\noindent {\em INDUCTIVE STEP:} Assume $P(k)$ to be true, for all $k < n$.
Consider a binary tree  formed by $n$ applications of the combination
rules. There are three cases.
\begin{enumerate}
\item The tree is a node with a left child that is a binary tree. 

Since the left child is a binary tree formed by $n-1$ applications of
the combination rules, by the inductive hypothesis, it has $1$ fewer
edges than nodes. Hence the whole tree has $1$ fewer edges than nodes,
too, since we've added one edge and one node.

\item The tree is a node with a right child that is a binary tree. 

This case is identical to the previous case.

\item The tree is a node with left and right binary-tree children. 

Then each of the two subtrees is formed by fewer than $n$ applications
of the combination rules. 
By inductive hypothesis (using strong induction here, since it needn't
be $n-1$), \\
each subtree has exactly $1$ fewer edges than nodes.
The whole tree adds $1$ node, the root node, and $2$ edges coming from it, 
and so also has exactly $1$ fewer edges than nodes. 
\end{enumerate}

This completes the inductive step and hence $P(n)$ is true for all
non-negative integers $n$.  This implies the result. 
\end{proof}

\section{The Well-Ordering Principle}

\subsection{Definition}

The {\em well-ordering principle} says: \\
Every nonempty set of natural numbers contains a least element.

This should be ``obvious'', as a property of natural numbers as
commonly understood. 
Well-ordering is not true for integers, positive reals, or positive
rationals. 

This may not look like an induction principle, but it's similar. 
Formally, it's equivalent (in the presence of basic axioms)
to induction (weak or strong). 
We'll come back to this later. 


\subsection{Example: lowest terms for rational numbers}
We left a slight hole in our proof that $\sqrt{2}$ is irrational -- 
we assumed that we could write a fraction in lowest terms, 
that it can be written in such a way that the numerator and denominator have no
common divisors. Now we can actually prove this fact:

\begin{theorem}
Every rational number can be written in lowest terms.
\end{theorem}

\begin{proof}
  Fix a rational $q$. \\ Let $S = \{b \in \naturals \mid (\exists a) q =
  a/b \}$.  (The set of possible denominators.) \\ $S$ is nonempty,
  because $q$ is, by definition, rational.  \\ Therefore $S$ has a
  least element $b$ and a corresponding $a$. \\ Then $a / b$ is a
  lowest-terms representation of $q$, (we still haven't proven it is
  unique, but we do know that $b$ is the smallest possible
  denominator).
\end{proof}

\section{Number Theory}

Having introduced the basic ideas of proofs, especially induction, we
now turn to introducing the basic discrete structures used in computer
science, such as natural numbers and integers, sets, relations, and functions,
equivalence relations, posets, and trees, graphs, and state machines. 

We'll then move on to analysis techniques (counting, recurrences, etc.). \\

Some definitions:
The natural numbers are $0, 1, \cdots$. \\
Integers also include negative values of the natural numbers. \\
The operations we will use are the usual arithmetic operations $+$, 
$\times$, etc. \\
The ordering is the usual order. \\

We will review some basic concepts: divisibility, primes, composite
numbers, modular arithmetic, etc., and give basic theorems about these notions,
and some simple algorithms based on these theorems. \\

For example:
\begin{itemize}
\item The fundamental theorem of arithmetic says that every number has a unique
factorization into primes. \\
\item Algorithm for finding GCD, the greatest common divisor of two numbers. \\
\item RSA algorithm for encrypting/decrypting data. \\
\end{itemize}

This subject matter, usually called {\em number theory}, used to be just for 
fun, but recently has become very important in
practice, mainly because of applications to security.
We will give some number theory examples using of well-ordering and
strong induction. 

\subsection{Example: Factorization}

\begin{theorem}
Every natural number $\geq 2$ can be factored as a product of primes.
\end{theorem}

\begin{proof}
Proof: By well-ordering.  \\
Let $S$ be the set of all natural numbers $\geq 2$ that cannot be
factored as a product of primes. \\
We will assume $S \ne \emptyset$ and derive a contradiction. \\
Since $S$ is not empty, well-ordering implies that it has a least
element, $n$. \\
This element $n$ cannot be a prime, because then it would be a
(trivial) product of primes. \\
So $n$ is composite -- write $n=ab$ where $1 < a,b <n$.  \\

Because $a,b < n$, and $n$ is the smallest element of $S$, it follows
that $a,b \notin S$. \\
This implies that 
$a$ can be written as a product of primes, say $p_1 p_2 \cdots p_k$, \\
and $b$ as a product of primes, say $q_1\cdots q_l$.  \\

Therefore, $n$ can be written as $p_1 \cdots p_k q_1 \cdots q_l$, \\
a product of primes, contradicting the claim that $n \in S$.  \\
 
Our assumption that $S \ne \emptyset$ must therefore be false (proof
by contradiction).
\end{proof}

It is also possible to prove this result using strong induction--
the argument is very similar, in fact.  (Check this.) 
This similarity should give you the idea that the well-ordering
principle is very similar to strong induction, in fact, it's equivalent 
(see later). \\

A fallacious argument: \\
Try to show that every number can be factored {\em uniquely} into primes. \\

Apply the same (well-ordering) proof as above, modifying the
definition of $S$: \\
Let $S$ be the set of all natural numbers $\geq 2$ that cannot be
factored {\em in exactly one way\/} as a product of primes.  \\

The fallacy is that even if $n = a b$ and $a$ and $b$ have unique
factorizations,
it is still possible that $n = c d$ for different $c$ and $d$, \\
producing a different factorization of $n$. \\

This argument is false, although the claim is true and is known as the
{\em fundamental theorem of arithmetic} (proof soon). \\


\subsection{The Division Algorithm}

(An old name, not really an algorithm).


\begin{theorem}
(The division algorithm.)  Let $a,b$ be nonzero naturals.  
Then $\exists! q,r$ such that  $0\le r < b$ and $a=qb+r$.
\end{theorem}

\begin{proof}
To prove this theorem, we need to prove both that some $r$ exists and
that this $r$ is unique (this is typical of unique-existence proofs). \\
Fix $a$, $b$. 

\begin{description}
\item[Existence:] 

Prove using well-ordering.  \\
Let $S = \{r \in \naturals \mid (\exists q \in \naturals) a = qb+r\}.$  \\

For example,
if $a = 13$ and $b = 3$, then $S = \{ 1, 4, 7, 10, 13  \}$. \\

$S$ is nonempty because $a = 0b + a$ implies $a \in S$.  
It follows by well-ordering that $S$ contains a least element $r_0$
such that $a = q_0 b + r_0$. 
We claim that this $(q_0,r_0)$ pair has all the properties we need. \\

The only interesting thing to check is that $r_0 < b$.
We can argue this by contradiction: suppose $r_0 \ge b$. 
Then $a = (q_0 + 1) b + (r_0 - b)$ and $r_0 - b \ge 0$, 
which implies that $r_0 - b \in S$. 
But this contradicts the assumption that $r_0$ is the least element of $S$.
Thus $r_0 < b$. \\

\item[Uniqueness:] 

Suppose $a = q_1 b + r_1 = q_2 b + r_2$. 
{\bf Without loss of generality}, suppose $r_1 > r_2$. 
(What we mean is, if $r_1 < r_2$, then we can swap $(q_1,r_1)$ 
with $(q_2,r_2)$ to make the assumption true.  It's a proof by cases,
but doing both cases at once.  Of course, we cannot
have $r_1 = r_2$.)  \\
Then subtract: 
$(q_1 - q_2) b = (r_1 - r_2)$. 
This means that $b \mid r_1 - r_2$. 
By assumption, $0 \leq r_1, r_2 < b$.
So $0 \leq r_1 - r_2 < b$. 
Therefore, $r_1 - r_2 = 0$.  \\
So $q_1 - q_2 = 0$.
\end{description}
\end{proof}

\begin{defin}
In the theorem above, $q$ is the {\em quotient} and $r$ is the {\em
remainder}.
\end{defin}

\subsection{Well-Ordering vs. Induction}

In the presence of basic axioms about numbers,
well-ordering turns out to be {\em equivalent} to induction (weak or
strong, since they are equivalent to each other). 

\begin{theorem}
Induction implies well-ordering.
\end{theorem}

\begin{proof}
We must show that every nonempty set contains a smallest element. \\
We can assume either kind of induction axiom, since they are
equivalent, so we'll use strong induction, since that gives us more power. 

Define predicate $P(n)$: 
Every set containing $n$ contains a smallest element. \\

If we prove that $P(n)$ is true for all $n$, then we have well-ordering 
since a nonempty set has to contain {\em some} least element $n$. \\


Prove this by strong induction: \\
Basis: $P(0)$: \\
If a set contains $0$, then $0$ is its smallest element. \\
Follows from basic ordering properties. \\

Inductive step: \\
Assume $(\forall k < n)P(k)$, and prove $P(n)$. \\
Consider a set containing $n$.  \\
If it contains no smaller element, then $n$ is the smallest element and
we are done.  \\
On the other hand, if it does contain a smaller element $k < n$, \\
then by the inductive hypothesis, it contains a smallest element and
again we are done. 
\end{proof}

\begin{theorem}
Well ordering implies induction. 
\end{theorem}

\begin{proof}
We will show it implies weak induction. 

We have to show that for any predicate $P$, 
$P(0)$ and $(\forall n) (P(n)\implies P(n+1))$ 
together imply $(\forall n) P(n)$.  

So fix $P$ and assume 
$P(0)$ and $(\forall n) (P(n) \implies P(n+1))$.

We have well-ordering, which says any nonempty $S$ has a smallest
element. 
We need to define the right $S$ so we can apply well-ordering: 
Define $S = \{n \mid \neg P(n)\}$. 
The set of elements not satisfying $P$. 

Suppose that $S \ne \emptyset$ and derive a contradiction. 

If $S \neq \emptyset$ then, by well-ordering, it contains a smallest
element $n$. 
Clearly $n \ne 0$ since $P(0)$ is true.  
Therefore $n-1$ is a natural number (uses basic order axioms) 
and it is not in $S$ (since $n$ is the smallest element). 
So $P(n-1)$ is true. 
It follows from weak induction that $P(n)$ is true. which contradicts
the claim that $n \in S$.  

We have derived a contradiction, meaning that our initial assumption, 
that $S$ is nonempty, is false. 
So $S$ is empty, which implies that $P(n)$ is true for all $n$, which is
what we needed to show.
\end{proof}


Remark:  If we redo the previous proof using strong induction
instead of weak, then it seems not to use the existence of a
predecessor!  This is a nicer proof, in fact. 

But this assumption is used elsewhere -- in the ``obvious''
implication that strong induction implies weak induction!

\subsection{Tiling}

Problem:  What 2-dimensional planes can be tiled using only $1 \times
3$ and $2 \times 2$ tiles?  Rotations are allowed. \\

Easy case:  What about $1 \times n$ and $n \times 1$? \\
We can tile these if and only if $n$ is a multiple of $3$. 
(We leave this to the reader to prove carefully.)  

Next case: $2 \times n$ (and $n \times 2$) \\
Claim any, if $n \geq 2$. 
Show by strong induction: \\

BASE: $2 \times 2$, one $2 \times 2$ tile.  \\
BASE: $2 \times 3$, two $1 \times 3$ tiles. \\

INDUCTIVE: $n \geq 4$. \\
If $n$ is even, then use one $2 \times 2$ tile. What's left is $2
\times (n-2)$, where $n - 2 \geq 2$.
So by strong inductive hypothesis, we can tile the rest. \\

If $n$ odd, then $n \geq 5$. \\
Then use two $1 \times 3$'s to reduce the area to $2 \times (n-3)$, where $n -
3 \geq 2$.  Again, strong inductive hypothesis gives the rest. 

Yet another case:  $3 \times n$ and $3 \times n$.\\
Fill the space with $1 \times 3$ tiles, obviously. \\

Finally:  What about $m \times n$, $m,n \geq 4$? \\
All $n$ can be tiled.
Proof by strong induction on $m$ where $P(m)$ says ``for all $n \geq
4$, can tile $m \times n$''. \\

Cases: \\
$m$ even, and $m \geq 4$: \\
The first two rows are tiled as in the $2 \times n$ case, if $n \geq 4$. it 
can be tiled by special case lemma. 
Remove these two columns, get $(m - 2) \times n$. 
If $m-2 = 2$ then can do rest by special case lemma,
else if $m - 2 \geq 4$, the rest can be tiled by the inductive hypothesis. \\

$m$ odd, and $m \geq 4$: \\
So $m \geq 5$. \\
Then first three rows can be done, by special case lemma. 
Removing these rows, we get $(m - 3) \times n$. 
If $m - 3 = 2$ then can do rest by special case lemma, 
else $m - 3 \geq 4$, and we can do rest by inductive hypothesis.

\end{document}

% Local Variables: 
% mode: latex
% TeX-master: t
% End: 
