\documentstyle[12pt,twoside,amsfonts,psfig]{article}
\input{macros-course-lectures}
\parskip1.5ex
\parindent0pt


\begin{document}
\lecture{notes15}{Notes 15}{October 29, 1996}

Readings: 4-1, 4-3, 5-4, 5-5 \\
Next: 4-3, 5-4 \\
After: 4-6, 4-7

\section{Introduction}

The next three lectures will present methods for {\em counting\/} the
number of elements in various sets. \\
Counting is one of the most important concepts of discrete math. \\
Subject called {\em combinatorics}. \\
\\
Counting arises in many different ways: \\
\\
Examples: Counting addresses, keys \\
How many phone numbers are available for use in the U.S.? \\
Internet addresses?  \\
200-digit primes for RSA cryptography? \\
\\
\\
Example: Measurement for programs \\
Counting amount of time, memory, disk space, etc., used by programs. \\
\\
\\
Example: Programs that enumerate and test \\
Some programs solve problems by enumerating all candidate solutions 
and trying them all. \\
To see which is best (optimization problem), or just to try to find
some solution. \\
\\
Operations research optimization problems: travelling salesman problem, etc. \\
Dynamic programming -- a clever method for cutting down the search space. \\
Circuit testing, by trying all inputs. \\
Finite-state program verification, by testing all reachable states. \\
\\
Nice to know how many candidates there are, so we know how long to
expect the program to run. \\
Also, counting methods often yield ways of explicitly {\em
enumerating\/} the elements. \\
Use the enumeration in the program. \\
\\
\\
Example: Probability \\
Want to know the probability that some event happens when some system
or program operates. \\
The probability of an event is closely related to the number of ways
that event might happen. \\
Calculating probabilities is based on counting number of ways. \\
\\
\\
You all understand counting informally. \\
Actually, it's a very formal notion, based on {\em bijections}: \\
A set $A$ has size (cardinality) $n$ if and only if there is a
bijection from $A$ to the set $\{1, \ldots, n \}$. \\
To prove this, we don't always exhibit a bijection directly to 
$\{1, \ldots, n \}$; sometimes, show a bijection to another set $B$,
which we already know is of size $n$. \\
Uses the fact that two sets have the same cardinality if and only if
there is a bijection between them. \\
\\
Often counting arguments will look quite informal, but can always
formalize them in terms of bijections. \\
\\
\\
Combinatorics contains a very large number of formulas to help in
counting. \\
It's impossible to learn them all. \\
The key is to learn the basics, then look up or derive the other
formulas when you need them. \\
\\
The complicated formulas can all be proved formally, using algebra. \\
Many of them can also be understood informally, using what we will
call a {\em thought experiment}. \\
This involves giving an intuitive meaning to the expressions on the
LHS and RHS of an equation, then verifying that these two meanings are
the same. 

\section{Basic Counting Principles}

Start with the basic counting principles: \\
Sum rule, inclusion-exclusion principle, product rule, tree diagrams, division rule \\

\subsection{Sum}

\begin{theorem}
\label{sum-disjoint-pair}
If  $A$ and $B$ are disjoint, then $|A \cup B| = |A| + |B|$.
\end{theorem}

This expresses the idea as an equation, which we will prove formally
in terms of bijections. \\
\\
Can also understand it by a ``thought experiment'': \\
LHS denotes the number of ways of choosing an element of $A \cup B$, \\
RHS denotes the number of ways of choosing an element of $A$ plus the
number of ways of choosing an element of $B$. \\
Since the sets are disjoint, the LHS is the sum...(informal)

\begin{proof}
Let $n_A = |A|$, $n_B = |B|$. \\
This means that there are bijections $f_A$ and $f_B$,
$f_A$ from $A$ to $\{1,\ldots,n_A\}$, $f_B$ from $B$ to $\{1,\ldots,n_B\}$.  \\
We want a bijection from $A \cup B$ to $\{1, \ldots, n_A + n_B \}$. \\
\\
So define a function $g$ on $A \cup B$ by: 
\[ g(a) = \left\{ \begin{array}{ll} f_A(a) & \mbox{ if } a \in A\\
 n_A+f_B(a) & \mbox{ if } a \in B \end{array}\right. \] 
The function $g$ is well-defined since $A$ and $B$ are disjoint. \\
We must show that $g$ is a bijection. \\
That is, one-to-one and onto. \\
\\
One-to-one: \\
Suppose that $g(a) = g(b)$. \\
If $a, b \in A$ then $g(a) = f_A(a) = f_A(b) = g(b)$; since $f_A$ is
one-to-one, we have $a = b$. \\
If $a,b \in B$ then $g(a) = n_A + f_B(a) = n_A + f_B(a) = g(b)$; so
$f_B(a) = f_B(b)$; since $f_B$ is one-to-one, we again have $a = b$. \\
If $a \in A$, $b \in B$ (wlog), then $g(a) \leq n_A$ and $g(b) > n_A$;
contradiction. \\
\\
Onto: \\
Let $n \in \{1, \ldots, n_A + n_B \}$. \\
If $n \leq n_A$ then $n = f_A(a)$ for some $a \in A$, since $f_A$ is
onto; so $n = g(a)$. \\
If $n > n_A$, then $n = n_A + m$ for some $m \in \{1, \ldots, n_B \}$;
then $m = f_B(b)$ for some $b \in B$, since $f_B$ is onto; so 
$g(b) = n_A + f_B(b) = n_A + m = n$.
\\
Therefore, $g$ is a bijection, as needed.
\end{proof}

Recall that any bijection has an inverse function. \\
Inverse here is a function from $\{1, \ldots, n_A + n_B \}$ to $A \cup
B$: \\
\[ g^{-1}(n) = \left\{ \begin{array}{ll} f_A^{-1}(n) & \mbox{ if } n \leq n_A\\
 f_B^{-1}(n-n_A) & \mbox{ if } n > n_A \end{array}\right. \]
\\
In fact, note that a function has an inverse if and only if it is a
bijection. \\
Therefore, showing that a function has an inverse is another way of
showing that it is a bijection. \\
Here, define the inverse function, must show it's defined everywhere,
and well-defined. 
\\
\begin{example}
99 math majors, 179 cs majors, \\
How many majors in math or cs (assume no double majors)? \\
99 + 179 = 278 
\end{example}

Extend sum rule to any number of sets:

\begin{theorem}
If  $A_1, \ldots, A_n$ are pairwise disjoint, then 
$|A_1 \cup \ldots \cup A_n| = |A_1| + |A_2| + \ldots + |A_n|$.
\end{theorem}

Proof by induction on $n$.

\begin{example}
99 math, 179 cs, 24 philosophy \\
Total in math or cs or philosophy = 302
\end{example}

\subsection{Inclusion-Exclusion}

What if the sets aren't disjoint? \\
E.g., math and cs majors, where some people are majoring in both?
\\
Consider two sets first. \\

\begin{theorem}
If $A$ and $B$ are any sets (not necessarily disjoint),
then $|A \cup B| = |A| + |B|- |A \cap B|$.
\end{theorem}

\begin{figure}[htbp]
\centerline{\psfig{figure=venn.ps,height=2in}}
\caption{Venn diagram.}
\label{fig:graph}
\end{figure}

\begin{proof}
(First proof)
Construct a bijection from $A \cup B$ to 
$\{1, \ldots, n_A + n_B - n_{AB}\}$ explicitly, mapping all the elements of
$A$ to $\{1, \ldots, n_A \}$, and all the elements of $B$ not in the
intersection to $\{n_A + 1, \ldots, n_A + n_B - n_{AB}\}$. \\
Here, $n_A, n_B, n_{AB}$ denote the number of elements in $A$, $B$ and
the intersection respectively.  There is a bijection from $B$ to $\{1,
\ldots, n_B\}$, so $B - (A \cap B)$ has a bijection to $\{1, \ldots,
n_B - n_{AB}\}$.  This assumes the elements in the intersection maps
to the largest elements.  We can then conver this to a bijection from
$B$ to $\{n_A + 1, \ldots , n_A + n_B - n_{AB}\}$.

\end{proof}

\begin{proof}
(Second proof)
Use Theorem \ref{sum-disjoint-pair}, for disjoint sets. \\
But apply it to the disjoint sets $A$ and $B - A$. \\
Since $A \cup B = A \cup (B - A)$, it's enough to find the cardinality
of $A \cup (B - A)$. \\
By Theorem \ref{sum-disjoint-pair}, this is $|A| + |B - A|$. \\
We want it to be $|A| + |B| - |B - A|$. \\
\\
So we need to show that $|B - A| = |B| - |A \cap B|$. \\
That is, show that $|B| = |B - A| + |A \cap B|$.  \\
But this follows again from Theorem \ref{sum-disjoint-pair}, this time
applied to $|B - A|$ and $|A \cap B|$ (which are disjoint and have $B$
as their union).
\end{proof}

The second proof above was constructed ``by need''.
A better way to write it, after you figure it out is in ordered steps,
beginning to end:
\begin{itemize}
\item
$B = (B - A) \cup (A \cap B)$, by set identity; disjoint.
\item
$|B| = |B - A| + |A \cap B|$, by Theorem \ref{sum-disjoint-pair}.
\item
$A \cup B = A \cup (B - A)$, by set identity; disjoint.
\item
$|A \cup B| = |A| + |B - A|$, by Theorem \ref{sum-disjoint-pair}.
\item
Now use algebra:
$|A \cup B| = |A| + |B - A| = |A| + |B| - |A \cap B|$.
\end{itemize}

The order in which you think of the steps the first time is often not
the same as the order in which the steps appear in the final version
of the proof (that builds logically, step-by-step).

\begin{example}
99 math majors, 179 cs majors, 7 majoring in both. \\
How many majors in math or cs? \\
99 + 179 - 7 = 271 
\end{example}

\begin{example}
How many numbers in $\{1,\ldots,100\}$ are divisible by $2$ or $5$?  
\begin{itemize}
\item $50$ divisible by $2$
\item $20$ divisible by $5$
\item $10$ divisible by both $2$ and $5$ 
\item total: $50+20-10=60$.
\end{itemize}
\end{example}

What about more than two sets? \\
What is $|A \cup B \cup C|$, in general? \\
Draw Venn diagram: \\
\\
Well, $|A \cup B \cup C| = |A| + |B| + |C|$ minus something for the
overlapping part. \\
What to remove for the overlapping part? \\
Counted each 2-overlapped part twice, and the 3-overlapped part 3
times. \\
Suppose we subtract $|A \cap B| + |B \cap C| + |A \cap C|$. \\
Now we're counting the 2-overlapped parts only once, as we want, but
the 3-overlapped part isn't counted at all! \\
So need to correct again, this time by {\em adding\/} in $|A \cap B \cap C|$, 
The full expression:

\begin{theorem}
$|A \cup B \cup C| = (|A| + |B| + |C|) - (|A \cap B| + |B \cap C| + |A
\cap C|) + |A \cap B \cap C|$.
\end{theorem}

\begin{example}
99 math, 179 cs, 24 philosophy \\
math and cs: 7 (includes all 3 also) \\
math and philo: 12 \\
cs and philo: 2 \\
all 3: 1 \\
Total in math or cs or philosophy $= (99 + 179 + 24) - (7 + 12 + 2) +
1 = 282$.
\end{example}

In general, for $m$ sets, alternate plus and minus terms:

\begin{theorem}
$|\cup_i A_i| = 
\Sigma_i |A_i| 
- \Sigma_{i,j \mbox{ distinct }} |A_i \cup A_j|
+ \Sigma_{i,j,k \mbox{ distinct }} |A_i \cup A_j \cup A_k|
- \ldots + (-1)^{m+1} |\cap A_i|$
\end{theorem}

Can be proved by induction on the number of sets (complicated details
-- LTTR). \\

But come back to this Thursday, with a ``thought experiment'' proof.

\begin{example}
How many primes less than $100$?  \\
$99$ minus numbers divisible by $2,3,5$ or $7$ (since non-prime means
divisivle by some prime $< \sqrt{100}$) plus these 4 numbers.
Let
\[
P_i = \mbox{numbers divisible by $i$}
\]
note $|P_i| = \lfloor 100/i \rfloor$.

\newcommand{\npart}[1]{{\lfloor \frac{100}{#1} \rfloor}}

Then quantity is:
\begin{eqnarray*}
99 & -&\npart{2}-\npart{3}-\npart{5}-\npart{7}\\
 &+&\npart{2\cdot 3} + \npart {2 \cdot 5} +\npart{2 \cdot 7} + \npart {3
  \cdot 5} +\npart{3 \cdot 7} +\npart {5\cdot 7}\\
&-&\npart{2 \cdot 3 \cdot 5} - \npart {2 \cdot 3 \cdot  7} - \npart{2
  \cdot 5 \cdot 7} - \npart{3 \cdot 5 \cdot 7}\\
&+&\npart{2 \cdot 3 \cdot 5 \cdot 7}
\end{eqnarray*}
which is
\[
99-50-33-20-14+16+10+7+6+4+2-3-2-1-0+0=21
\]
so there are 25 primes. \\

\end{example}

\subsection{Product}

\begin{theorem}
If $A$ and $B$ are any sets, then
$| A \times B| = |A| \cdot |B|$.
\end{theorem}

Thought experiment: \\
LHS represents the number of ways of choosing a pair consisting of an
element of $A$ and an element of $B$. \\
RHS represents number of ways to choose an element of $A$, multiplied
by number of ways to choose an element of $B$. \\
If there are $n_A$ ways to choose an element of $A$, and $n_B$ ways to
choose an element of $B$, then there are $n_A \cdot n_B$ ways to
choose a pair. 

\begin{proof}
Let $n_A = |A|$, $n_B = |B|$. \\
Let $f_A$ be a bijection from $A$ to $\{0,\ldots,n_A-1\}$, 
$f_B$ from $B$ to $\{0,\ldots,n_B - 1\}$.  \\
These exist since these sets are the same size as the corresponding
sets starting at $1$. \\
We want a bijection from $A \times B$ to 
$\{0, \ldots, n_A \cdot n_B -1 \}$.
\\
Define by: \\
\[ g((a,b)) = n_B(f_A(a)) + f_B(b) \]  
We can show that $g$ is a bijection with inverse: 
\[ g^{-1}(n) = (f_A^{-1}(\lfloor\frac{n}{n_B}\rfloor),
f_B^{-1}(n \mbox{ mod } n_B)).\]
In other words, for any $n \in \{ 0, \ldots, n_A \cdot n_B - 1 \}$,
write $n = q n_B + r$, \\
where $0 \leq r < n_B$, by division theorem. \\
Then 
\[ g^{-1}(n) = (f_A^{-1}(q), f_B^{-1}(r)).\]
\end{proof}

\begin{example}
How many ways can you choose $2$ courses in different subjects from
among $5$ Math, $7$ CS, and $10$ EE courses?
\begin{itemize}
\item $1$ Math and $1$ CS: $5\times 7=35$ ways (product rule)
\item $1$ Math and $1$ EE: $5 \times 10=50$ ways
\item $1$ CS and $1$ EE: $7 \times 10=70$ ways.
\end{itemize}
By sum rule, total of $155$ ways.
\end{example}

Extend to $m$ sets:

\begin{theorem}
$|A_1 \times A_2 \times \ldots \times A_m| =
|A_1| \cdot |A_2| \cdot \ldots \cdot |A_m|$

This says that
$m$ successive choices, with exactly $n_k$ choices possible at the
$k^{th}$ step, can produce a total of exactly $n_1n_2\cdots n_m$ different
results.
\end{theorem}

\begin{example}
How many binary numbers of length n?  \\
By product rule, $2 \times \ldots \times 2$ ($n$ terms). \\
So $2^n$.
\end{example}

\begin{example}
Number of subsets of $n$-element set? \\
We already know this is $2^n$. \\
Each subset corresponds to a bit string of length $n$, with a $1$ in
position $k$ meaning that the $k$th element is in the subset. \\
That is, set of subsets has bijection to set of bit strings of length
$n$, which has $2^n$ elements.
\end{example}

\begin{example}
License plates -- 3 letters followed by 3 digits:
$26 \times 26 \times 26 \times 10 \times 10 \times 10$.
\end{example}

\begin{example}
Number of passwords -- length $8$, letters and digits, at least one
letter. \\
First calculate number of length $8$ strings of letters and digits,
counting those witn no letters: $36^8$. \\
Then subtract those with no letters: $26^8$. \\
The difference is what's needed: $36^8 - 26^8$.
\end{example}

\begin{example}
How many functions from set $A$ with $m$ elements to set $B$ with $n$
elements? \\
Function corresponds to $m$ choices from $B$, one for each element of
$A$.  \\
That is, set of functions has bijection to set $B \times \ldots \times
B$ (m terms). \\
This set has size $n^m$, by the product rule.
\end{example}

\begin{example}
How many length-$n$ palindromes over $\{ 0, 1 \}$? \\
Turns out exactly $2^{\lceil n/2 \rceil}$.

\begin{proof} We consider the cases $n$ even and odd separately.
\begin{itemize}
\item If $n$ is even we  define a bijection from the set 
$\{0,1\}^{n/2}$ to the set of palindromes of length $n$. This will
imply that there are $|\{0,1\}^{n/2}|=2^{n/2}$ palindromes. For a
sequence $a=(a_1,\cdots,a_{n/2})$ define
$f(a)=(a_1,\cdots,a_{n/2},a_{n/2},\cdots,a_1)$. Clearly, $f(a)$ is a
palindrome and $f$ is bijective.  
\item if $n$ is odd, we define a bijection from the set 
$\{0,1\}^{\lceil n/2\rceil}$ to the set of palindromes of length
$n$. This will imply that there are $|\{0,1\}^{\lceil n/2\rceil}|=2^{\lceil n/2\rceil}$
palindromes. For a sequence $a=(a_1,\cdots,a_{\lceil n/2\rceil})$ define
$g(a)=(a_1,\cdots,a_{\lceil n/2\rceil},\cdots,a_1)$. Clearly, $g(a)$ is a
palindrome and $g$ is bijective. 
\end{itemize}
Hence, in either case there are $2^{\lceil n/2 \rceil}$ palindromes.
\end{proof}
\end{example}

\subsection{Tree Representation}

Interpreting product rule in terms of number of successive choices.
Can represent successive choices by tree, then total number of choices
is number of leaves. 

\begin{example}
Bit strings of length 3:  Draw tree
\end{example}

\begin{example}
License places
\end{example}

This tree representation can also be used to count numbers of
successive choices, in the more general case where later choices are
from different sets, depending on the results of the earlier choices. \\
Then the answer isn't a product, but can still be calculated from the
number of leaves of the tree. 

\begin{example}
$3$ blue shirts, $2$ red shirts, $1$ green shirt, 
$2$ grey pants and $3$ brown pants.  
How many outfits if grey matches red/blue and brown matches red/green?
\begin{itemize}
\item $6$ blue-grey, $4$ red-grey, $6$ red-brown and $3$ green-brown makes 19.
\item Alternatively, the tree of outcomes may be represented as below (See Fig. 1):
\begin{figure}[htbp]
\centerline{\psfig{figure=choice.ps,height=2in}}
\caption{Tree of outcomes.}
\label{fig:grap}
\end{figure}

\end{itemize}
\end{example}

\begin{example}
Number of strings of letters of length 5, with no repeats: \\
$26 \times 25 \times 24 \times 23 \times 22$. \\
Show as tree, with later sets depending on earlier choices.
\end{example}

\subsection{Division Rule}

One last basic rule: division rule:

\begin{theorem}
If there is a $k$ to $1$ mapping from $A$ to $B$,
and $|A| = n$, then $|B| = n/k$. 
\end{theorem}

\begin{example}
How many subsets of size $2$ are there, of a $5$-element set $A$? \\
Let $S$ be this set of subsets. \\
\\
First, consider set $P$ of ordered pairs without repeats. \\
Have $|P| = 5 \times 4$ (since can't repeat -- consider tree). \\
\\
There's a $2$ to $1$ mapping from $P$ to $S$ -- just ignore the order.\\
So $S$ has $20 / 2 = 10$ elements.
\end{example}

\section{Permutations and Combinations}

Some of the examples we have seen (like the last one) deal with the
number of ways of choosing some particular number of elements from a set. \\
As you can see in this last example, there are two different ways to
select them: in a particular order, or just in an (unordered) set. \\
Selections are called {\em permutations\/} when we care about the
order, and {\em combinations} when we don't.

\subsection{Permutations}

\begin{defin}
An {\em $r$-permutation} of a set of $n$ elements is an ordered
arrangement of $r$ of the $n$ elements (all distinct).
\end{defin}

\begin{example}
$S=\set{A,B,C}$. \\
The $2$-permutations are $(A,B),(B,A),(A,C),(C,A),(B,C),(C,B)$.
\\
$6$ $2$-permutations
\end{example}

How many $r$-permutations of a set of $n$ elements?
\begin{itemize}
\item $n$ ways to choose first element
\item given first choice, $n-1$ ways to choose second element
\item $n-2$ to choose third
\item overall: 
\[ n(n-1)\cdots(n-r+1) = \frac{n!}{(n-r)!}.
\]
\end{itemize}
Denote this by $P(n,r)$.

\begin{defin} 
A {\em permutation} of $n$ elements is an $n$-permutation of them. 
\end{defin}

$P(n,n)=n!$

\begin{example}
How many ways can $n$ people stand in a circle?
\begin{itemize}
\item Map each permutation to a ring.
\item $n$ permutations mapped to each ring.
\item By division rule, number of rings is $n!/n = (n-1)!$.
\end{itemize}
\end{example}

\subsection{Combinations}

\begin{defin}
An {\em $r$-combination} of a set of $n$ elements is an (unordered)
subset of the $n$ elements.
\end{defin}

\begin{example}
$S=\set{A,B,C}$. \\
The $2$-combinations are $\set{A,B},\set{A,C},\set{B,C}$. \\
$3$ of them. 
\end{example}

How many $r$-combinations of a set of $n$ elements? 
\begin{itemize}
\item Map each $r$-permutation to an $r$-combination by ignoring the order.
\item How many $r$-permutations mapped to each $r$-combination? $r!$ - because
there are $P(r,r) = r!$ permutations of a given $r$-combination.
\item By division rule, $P(n,r)/r!$
\end{itemize}
Denote this by $C(n,r)$, or more commonly as ${n \choose r}$.

\begin{theorem} 
The number of $r$-combinations of an $n$-set is: 
\[
{n \choose r} = \frac{n!}{r!(n-r)!}
\]
\end{theorem}
${n \choose r}$ is called a {\em binomial coefficient} for reasons
that will become clear later.

\begin{example}
How many ways can $3$ distinct numbers be chosen from $1,\ldots,10$?
\\
This is ${10 \choose 3} = \frac{10!}{3!7!} = 120$.
\end{example}

\begin{example}
How many ways can $3$ distinct numbers be chosen from $1,\ldots,10$ so
that their sum is divisible by 3?
\begin{itemize}
\item $A=\set{3,6,9}$ ($=0 \bmod 3$)
\item $B=\set{1,4,7,10}$ ($=1\bmod 3$)
\item $C=\set{2,5,8}$ ($=2 \bmod 3$)
\end{itemize}
Need sum to be zero mod $3$.  If any two are equal $\bmod 3$, then
the third must also be equal.  So they are all same or all
different.  Thus the possibilities are:
\begin{itemize} 
\item 3 from $A$: ${3 \choose 3}=1$.
\item 3 from $B$: ${4 \choose 3} = 4$.
\item 3 from $C$: ${3 \choose 3} = 1$.
\item One from each: $3 \cdot 4 \cdot 3 = 36$ (product rule).
\end{itemize}
Total: 42.
\end{example}

\end{document}


Problems on Lectures 15 and 16:

Section 4.1: 
Ex. 18, 30, 34, 38, 42, 44  (basic -- select problems)

Section 4.3:
Ex. 16, 20, 28 (simple permutations and combinations -- select problems)

Ex. 42 (binomial theorem)

Ex. 52 (binomial identity, combinatorial proof)
Is there a reasonably-difficulty algebraic proof here?
I worked on it for a few minutes and didn't get it, but there should
be something not too hard...

Section 5.4:
Ex. 22 (inductive proof of inclusion-exclusion)

Section 5.5:
Ex. 6 (application of inclusion-exclusion)

