\documentstyle[12pt,twoside,amsfonts,psfig]{article}
\input{macros-course-lectures}
\parskip1.5ex
\parindent0pt


\begin{document}
\lecture{notes17}{Basic Combinatorics}{April 15, 1997}

Readings: 4-1, 4-3, 5-4, 5-5 \\
Next: 4-3, 5-4 \\
After: 4-6, 4-7

Change of plan. No homework due next tuesday.  This week's material
will be lightly covered on exam.  pset out thursday, due week after
quiz.


\section{Introduction}

The next lectures will present methods for {\em counting\/} the
number of elements in various sets. 
Counting is one of the most important concepts of discrete math. 
The subject called {\em combinatorics}. 

Counting arises in many different ways: 

Examples: Counting addresses, keys \\
How many phone numbers are available for use in the U.S.? 
Internet addresses?  
200-digit primes for RSA cryptography? 


Example: Measurement for programs \\
Counting amount of time, memory, disk space, etc., used by programs. 


Example: Programs that enumerate and test \\
Some programs solve problems by enumerating all candidate solutions 
and trying them all. 
To see which is best (optimization problem), or just to try to find
some solution. 

Operations research optimization problems: travelling salesman problem, etc. \\
Dynamic programming -- a clever method for cutting down the search space. \\
Circuit testing, by trying all inputs. \\
Finite-state program verification, by testing all reachable states. \\

It's nice to know how many candidates there are, so we know how long to
expect the program to run. 
Also, counting methods often yield ways of explicitly {\em
enumerating\/} the elements, so that you can 
use the enumeration in the program. 


Example: Probability \\
We want to know the probability that some event happens when some system
or program operates. 
The probability of an event is closely related to the number of ways
that event might happen. 
Calculating probabilities is based on counting number of ways. 

You all understand counting informally. 
Recall that it's actually a very formal notion, based on {\em bijections}: \\
A set $A$ has size (cardinality) $n$ if and only if there is a
bijection from $A$ to the set $\{1, \ldots, n \}$. 
To prove this, we don't always exhibit a bijection directly to 
$\{1, \ldots, n \}$; or sometimes, show a bijection to another set $B$,
which we already know is of size $n$. 
This uses the fact that two sets have the same cardinality if and only if
there is a bijection between them. 

Often counting arguments look quite informal, but we can always
formalize them in terms of bijections. 


Combinatorics contains a very large number of formulas to help in
counting, and 
it's impossible to learn them all. 
The key is to learn the basics, then look up or derive the other
formulas when you need them. 

The complicated formulas can all be proven formally using algebra. 
Many of them can also be understood informally, using what we will
call a {\em thought experiment}. 
This involves giving an intuitive meaning to the expressions on the
left-hand side (LHS) and right-hand side (RHS) of an equation, then verifying that these two meanings are
the same. 

\section{Basic Counting Principles}

Start with the basic counting principles: \\ Sum rule,
inclusion-exclusion principle, product rule, case analysis, division
rule.

\subsection{Sum}

\begin{theorem}
\label{sum-disjoint-pair}
If  $A$ and $B$ are disjoint, then $|A \cup B| = |A| + |B|$.
\end{theorem}

This expresses the idea as an equation, which we will prove formally
in terms of bijections. 

We can also understand it by a ``thought experiment'': \\
The LHS denotes the number of ways of choosing an element of $A \cup B$, \\
The RHS denotes the number of ways of choosing an element of $A$ plus the
number of ways of choosing an element of $B$. \\
Since the sets are disjoint, the LHS is the sum...(informal)

\begin{proof}
Let $n_A = |A|$, $n_B = |B|$. \\
This means that there are bijections $f_A$ and $f_B$,
$f_A$ from $A$ to $\{1,\ldots,n_A\}$, $f_B$ from $B$ to $\{1,\ldots,n_B\}$.  \\
We want a bijection from $A \cup B$ to $\{1, \ldots, n_A + n_B \}$. 

So define a function $g$ on $A \cup B$ by: 
\[ g(a) = \left\{ \begin{array}{ll} f_A(a) & \mbox{ if } a \in A\\
 n_A+f_B(a) & \mbox{ if } a \in B \end{array}\right. \] 
The function $g$ is well-defined since $A$ and $B$ are disjoint. \\
We must show that $g$ is a bijection, so we must prove it's 
one-to-one and onto. 

\noindent One-to-one: \\
Suppose that $g(a) = g(b)$. \\
If $a, b \in A$ then $g(a) = f_A(a) = f_A(b) = g(b)$; since $f_A$ is
one-to-one, we have $a = b$. \\
If $a,b \in B$ then $g(a) = n_A + f_B(a) = n_A + f_B(a) = g(b)$; so
$f_B(a) = f_B(b)$; since $f_B$ is one-to-one, we again have $a = b$. \\
If $a \in A$, $b \in B$ (wlog), then $g(a) \leq n_A$ and $g(b) > n_A$;
contradiction. 

\noindent Onto: \\
Let $n \in \{1, \ldots, n_A + n_B \}$. \\
If $n \leq n_A$ then $n = f_A(a)$ for some $a \in A$, since $f_A$ is
onto; so $n = g(a)$. \\
If $n > n_A$, then $n = n_A + m$ for some $m \in \{1, \ldots, n_B \}$;
then $m = f_B(b)$ for some $b \in B$, since $f_B$ is onto; so 
$g(b) = n_A + f_B(b) = n_A + m = n$.
\\
Therefore, $g$ is a bijection, as needed.
\end{proof}

Recall that any bijection has an inverse function. \\
Inverse here is a function from $\{1, \ldots, n_A + n_B \}$ to $A \cup
B$: \\
\[ g^{-1}(n) = \left\{ \begin{array}{ll} f_A^{-1}(n) & \mbox{ if } n \leq n_A\\
 f_B^{-1}(n-n_A) & \mbox{ if } n > n_A \end{array}\right. \]

In fact, note that a function has an inverse if and only if it is a
bijection. 
Therefore, showing that a function has an inverse is another way of
showing that it is a bijection. 
Here, define the inverse function, show it's defined everywhere,
and well-defined. 

\begin{example}
99 math majors, 179 CS majors, \\
How many majors in math or CS (assume no double majors)? \\
99 + 179 = 278 
\end{example}

Extend sum rule to any number of sets:

\begin{theorem}
If  $A_1, \ldots, A_n$ are pairwise disjoint, then 
$|A_1 \cup \ldots \cup A_n| = |A_1| + |A_2| + \ldots + |A_n|$.
\end{theorem}

Proof by induction on $n$.

\begin{example}
99 math, 179 CS, 24 philosophy \\
Total in math or CS or philosophy = 302
\end{example}

\subsection{Inclusion-Exclusion}

What if the sets aren't disjoint? 
E.g., math and CS majors, where some people are majoring in both?

Consider two sets first:

\begin{theorem}
If $A$ and $B$ are any sets (not necessarily disjoint),
then $|A \cup B| = |A| + |B|- |A \cap B|$.
\end{theorem}

\begin{figure}[htbp]
\centerline{\psfig{figure=venn.ps,height=2in}}
\caption{Venn diagram.}
\label{fig:graph}
\end{figure}

\begin{proof}
Use Theorem \ref{sum-disjoint-pair}, for disjoint sets. \\
Note that
\begin{eqnarray*}
A \cup B &= &(A-B) \cup (A \cap B) \cup (B-A)\\
A &= &(A-B) \cup (A \cap B)\\
B &= &(B-A) \cup (A \cap B)\\
\end{eqnarray*}
and that all these are expressed as disjoint unions (can prove by set
theory, obvious by venn diagram).  Thus, by disjoint sum rule,
\begin{eqnarray*}
|A \cup B| &= &|A-B| + |A \cap B| + |B - A|\\
|A| &= &|A-B| + |A\cap B|\\
|B| &= &|B-A| + |A \cap B|
\end{eqnarray*}
subtracting second and third equation from first gives result.
\end{proof}


\begin{example}
99 math majors, 179 CS majors, 7 majoring in both. \\
How many majors in math or CS? \\
99 + 179 - 7 = 271 
\end{example}

\begin{example}
How many numbers in $\{1,\ldots,100\}$ are divisible by $2$ or $5$?  
\begin{itemize}
\item $50$ divisible by $2$
\item $20$ divisible by $5$
\item $10$ divisible by both $2$ and $5$ 
\item total: $50+20-10=60$.
\end{itemize}
\end{example}

What about more than two sets? \\
What is $|A \cup B \cup C|$, in general? \\

\begin{figure}[htbp]
\centerline{\psfig{figure=venn2.ps,height=3.5in}}
\caption{Venn diagram.}
\label{fig:graph}
\end{figure}


Well, $|A \cup B \cup C| = |A| + |B| + |C|$ minus something for the
overlapping part. 
What should we remove for the overlapping part? 
We counted each 2-overlapped part twice, and the 3-overlapped part 3
times. 
Suppose we subtract $|A \cap B| + |B \cap C| + |A \cap C|$. 
Now we're counting the 2-overlapped parts only once, as we want, but
the 3-overlapped part isn't counted at all! 
So need to correct again, this time by {\em adding\/} in $|A \cap B \cap C|$, 
The full expression:

\begin{theorem}
$|A \cup B \cup C| = (|A| + |B| + |C|) - (|A \cap B| + |B \cap C| + |A
\cap C|) + |A \cap B \cap C|$.
\end{theorem}

\begin{example}
99 math, 179 CS, 24 philosophy \\
math and CS: 7 (includes someone in all 3 also) \\
math and philo: 12 \\
CS and philo: 2 \\
all 3: 1 \\
Total in math or CS or philosophy $= (99 + 179 + 24) - (7 + 12 + 2) +
1 = 282$.
\end{example}

In general, for $m$ sets, alternate plus and minus terms:

\begin{theorem}
$|\cup_i A_i| = 
\Sigma_i |A_i| 
- \Sigma_{i,j \mbox{ distinct }} |A_i \cup A_j|
+ \Sigma_{i,j,k \mbox{ distinct }} |A_i \cup A_j \cup A_k|
- \ldots + (-1)^{m+1} |\cap A_i|$
\end{theorem}

Can be proved by induction on the number of sets (we'll see this later).
We'll come back to this Thursday, with a ``thought experiment'' proof.

\begin{example}
How many primes less than $100$?  \\
$99$ minus numbers divisible by $2,3,5$ or $7$ (since non-prime means
divisible by some prime $< \sqrt{100}$) plus these 4 numbers.
Let
\[
P_i = \mbox{numbers divisible by $i$}
\]
note $|P_i| = \lfloor 100/i \rfloor$.

\newcommand{\npart}[1]{{\lfloor \frac{100}{#1} \rfloor}}

Then quantity is:
\begin{eqnarray*}
99 & -&\npart{2}-\npart{3}-\npart{5}-\npart{7}\\
 &+&\npart{2\cdot 3} + \npart {2 \cdot 5} +\npart{2 \cdot 7} + \npart {3
  \cdot 5} +\npart{3 \cdot 7} +\npart {5\cdot 7}\\
&-&\npart{2 \cdot 3 \cdot 5} - \npart {2 \cdot 3 \cdot  7} - \npart{2
  \cdot 5 \cdot 7} - \npart{3 \cdot 5 \cdot 7}\\
&+&\npart{2 \cdot 3 \cdot 5 \cdot 7}
\end{eqnarray*}
which is
\[
99-50-33-20-14+16+10+7+6+4+2-3-2-1-0+0=21
\]
so there are 25 primes.

\end{example}

\subsection{Product}

\begin{theorem}
If $A$ and $B$ are any sets, then
$| A \times B| = |A| \cdot |B|$.
\end{theorem}

Thought experiment: \\
LHS represents the number of ways of choosing a pair consisting of an
element of $A$ and an element of $B$. \\
RHS represents number of ways to choose an element of $A$, multiplied
by number of ways to choose an element of $B$. \\
If there are $n_A$ ways to choose an element of $A$, and $n_B$ ways to
choose an element of $B$, then there are $n_A \cdot n_B$ ways to
choose a pair. 

\begin{proof}
Let $n_A = |A|$, $n_B = |B|$. \\
Let $f_A$ be a bijection from $A$ to $\{0,\ldots,n_A-1\}$, 
$f_B$ from $B$ to $\{0,\ldots,n_B - 1\}$.  \\
These exist since these sets are the same size as the corresponding
sets starting at $1$. \\
We want a bijection from $A \times B$ to 
$\{0, \ldots, n_A \cdot n_B -1 \}$.
\\
Define by: \\
\[ g(a,b) = n_B(f_A(a)) + f_B(b) \]  
We can show that $g$ is a bijection with inverse: 
\[ g^{-1}(n) = (f_A^{-1}(\lfloor\frac{n}{n_B}\rfloor),
f_B^{-1}(n \mbox{ mod } n_B)).\]
In other words, for any $n \in \{ 0, \ldots, n_A \cdot n_B - 1 \}$,
write $n = q n_B + r$, \\
where $0 \leq r < n_B$, by division theorem. \\
Then 
\[ g^{-1}(n) = (f_A^{-1}(q), f_B^{-1}(r)).\]
\end{proof}

\begin{example}
How many ways can you choose $2$ courses in different subjects from
among $5$ Math, $7$ CS, and $10$ EE courses?
\begin{itemize}
\item $1$ Math and $1$ CS: $5\times 7=35$ ways (product rule)
\item $1$ Math and $1$ EE: $5 \times 10=50$ ways
\item $1$ CS and $1$ EE: $7 \times 10=70$ ways.
\end{itemize}
By sum rule, total of $155$ ways.
\end{example}

Extend to $m$ sets:

\begin{theorem}
$|A_1 \times A_2 \times \ldots \times A_m| =
|A_1| \cdot |A_2| \cdot \ldots \cdot |A_m|$

This says that
$m$ successive choices, with exactly $n_k$ choices possible at the
$k^{th}$ step, can produce a total of exactly $n_1n_2\cdots n_m$ different
results.
\end{theorem}

\begin{example}
How many binary numbers of length n?  \\
By product rule, $2 \times \ldots \times 2$ ($n$ terms). 
So $2^n$.
\end{example}

\begin{example}
Number of subsets of $n$-element set? \\
We already know this is $2^n$. 
Each subset corresponds to a bit string of length $n$, with a $1$ in
position $k$ meaning that the $k$th element is in the subset. 
That is, set of subsets has bijection to set of bit strings of length
$n$, which has $2^n$ elements.
\end{example}

\begin{example}
License plates -- 3 letters followed by 3 digits:
$26 \times 26 \times 26 \times 10 \times 10 \times 10$.
\end{example}

\begin{example}
Number of passwords -- length $8$, letters and digits, at least one
letter. \\
First calculate number of length $8$ strings of letters and digits,
counting those with no letters: $36^8$. \\
Then subtract those with no letters: $10^8$. \\
The difference is what's needed: $36^8 - 10^8$.
\end{example}

\begin{example}
How many functions from set $A$ with $m$ elements to set $B$ with $n$
elements? \\
Function corresponds to $m$ choices from $B$, one for each element of
$A$.  \\
That is, set of functions has bijection to set $B \times \ldots \times
B$ (m terms). \\
This set has size $n^m$, by the product rule.
\end{example}

\begin{example}
How many length-$n$ palindromes over $\{ 0, 1 \}$? \\
Turns out exactly $2^{\lceil n/2 \rceil}$.

\begin{proof} We consider the cases $n$ even and odd separately.
\begin{itemize}
\item If $n$ is even we  define a bijection from the set 
$\{0,1\}^{n/2}$ to the set of palindromes of length $n$. This will
imply that there are $|\{0,1\}^{n/2}|=2^{n/2}$ palindromes. For a
sequence $a=(a_1,\cdots,a_{n/2})$ define
$f(a)=(a_1,\cdots,a_{n/2},a_{n/2},\cdots,a_1)$. Clearly, $f(a)$ is a
palindrome and $f$ is bijective.  
\item if $n$ is odd, we define a bijection from the set 
$\{0,1\}^{\lceil n/2\rceil}$ to the set of palindromes of length
$n$. This will imply that there are $|\{0,1\}^{\lceil n/2\rceil}|=2^{\lceil n/2\rceil}$
palindromes. For a sequence $a=(a_1,\cdots,a_{\lceil n/2\rceil})$ define
$g(a)=(a_1,\cdots,a_{\lceil n/2\rceil},\cdots,a_1)$. Clearly, $g(a)$ is a
palindrome and $g$ is bijective. 
\end{itemize}
Hence, in either case there are $2^{\lceil n/2 \rceil}$ palindromes.
\end{proof}
\end{example}

\subsection{Case Analysis and its Tree Representation}

If we interpret product rule in terms of number of successive choices,
and can represent the successive choices by tree, then the total number 
of choices is the number of leaves. 

\begin{example}
Bit strings of length 3:  Draw tree
\begin{figure}[htbp]
\centerline{\psfig{figure=tree.ps,height=3in}}
\caption{Tree of choices}
\label{fig:graph}
\end{figure}

\end{example}

Note that the tree is a kind of recursive sum rule: the top level is summing
over possible first bits, etc.

This tree representation can also be used to count the numbers of
successive choices. In the more general case, the  choices  come
from different sets, depending on the results of the earlier choices. 
In this case the answer isn't a product, but it can still be calculated from the
number of leaves of the tree. 

\begin{example}
$3$ blue shirts, $2$ red shirts, $1$ green shirt, 
$2$ grey pants and $3$ brown pants.  
How many outfits if grey matches red/blue and brown matches red/green?
\begin{itemize}
\item $6$ blue-grey, $4$ red-grey, $6$ red-brown and $3$ green-brown makes 19.
\item Alternatively, the tree of outcomes may be represented as below (See Fig. 1):
\begin{figure}[htbp]
\centerline{\psfig{figure=choice.ps,height=2in}}
\caption{Tree of outcomes.}
\label{fig:grap}
\end{figure}

\end{itemize}
\end{example}

\begin{example}
Number of strings of letters of length 5, with no repeated letters: \\
$26 \times 25 \times 24 \times 23 \times 22$. \\
Show as tree, with later sets depending on earlier choices.
\end{example}

\subsection{Division Rule}

One last basic rule: division rule:

\begin{theorem}
If there is a $k$ to $1$ mapping from $A$ to $B$,
and $|A| = n$, then $|B| = n/k$. 
\end{theorem}

\begin{example}
How many subsets of size $2$ are there, of a $5$-element set $A$? \\
Let $S$ be this set of subsets. 

First, consider the set $P$ of ordered pairs without repeats. 
We have $|P| = 5 \times 4$ (since elements can't repeat -- consider a tree). 

There's a $2$ to $1$ mapping from $P$ to $S$ -- just ignore the order. \\
So $S$ has $20 / 2 = 10$ elements.
\end{example}

\subsection{A Remark on ``Distinct Ways''}

Counting problems are often formulated as ``how many distinct ways can
you$\ldots$'' and this is a somewhat tricky notion.  ``Distinct''
means whatever the poser of the problem wants it to mean.   In the
clothing example above, I assumed that the 2 red shirts were

distinct.  I could have formulated a question asking only
about distinct color combinations of clothes---then the answer would
be 4, even though the experiment (picking clothes) would be the same.


\section{Permutations and Combinations}

Some of the examples we have seen (like the last one) deal with the
number of ways of choosing some particular number of elements from a set. 
As you can see in this last example, there are two different ways to
select them: in a particular order, or just in an (unordered) set. 
Selections are called {\em permutations\/} when we care about the
order, and {\em combinations} when we don't.

\subsection{Permutations}

\begin{defin}
An {\em $r$-permutation} of a set of $n$ elements is an ordered
arrangement of $r$ of the $n$ elements (all distinct).
\end{defin}

\begin{example}
$S=\set{A,B,C}$. 
The $2$-permutations are $(A,B),(B,A),(A,C),(C,A),(B,C),(C,B)$.

There are $6$ $2$-permutations
\end{example}

How many $r$-permutations of a set of $n$ elements?
\begin{itemize}
\item $n$ ways to choose the first element
\item given first choice, $n-1$ ways to choose the second element
\item $n-2$ to choose the third element
\item overall: 
\[ n(n-1)\cdots(n-r+1) = \frac{n!}{(n-r)!}.
\]
\end{itemize}
Denote this by $P(n,r)$.

\begin{defin} 
A {\em permutation} of $n$ elements is an $n$-permutation of them. 
\end{defin}

$P(n,n)=n!$

\begin{example}
How many ways can $n$ people stand in a circle?
\begin{itemize}
\item Map each permutation to a ring.
\item There are $n$ permutations mapped to each ring.
\item By the division rule, the number of rings is $n!/n = (n-1)!$.
\end{itemize}
\end{example}

\subsection{Combinations}

\begin{defin}
An {\em $r$-combination} of a set of $n$ elements is an (unordered)
subset of the $n$ elements.
\end{defin}

\begin{example}
$S=\set{A,B,C}$. 
The $2$-combinations are $\set{A,B},\set{A,C},\set{B,C}$. 
There are $3$ of them. 
\end{example}

How many $r$-combinations of a set of $n$ elements? 
\begin{itemize}
\item Map each $r$-permutation to an $r$-combination by ignoring the order.
\item How many $r$-permutations mapped to each $r$-combination? $r!$ - because
there are $P(r,r) = r!$ permutations of a given $r$-combination.
\item By division rule, $P(n,r)/r!$
\end{itemize}
We denote this by $C(n,r)$, or more commonly as ${n \choose r}$.

\begin{theorem} 
The number of $r$-combinations of an $n$-set is: 
\[
{n \choose r} = \frac{n!}{r!(n-r)!}
\]
\end{theorem}
${n \choose r}$ is called a {\em binomial coefficient} for reasons
that will become clear later.

\begin{example}
How many ways can $3$ distinct numbers be chosen from $1,\ldots,10$?

This is ${10 \choose 3} = \frac{10!}{3!7!} = 120$.
\end{example}

\begin{example}
How many ways can $3$ distinct numbers be chosen from $1,\ldots,10$ so
that their sum is divisible by 3?
\begin{itemize}
\item $A=\set{3,6,9}$ ($=0 \bmod 3$)
\item $B=\set{1,4,7,10}$ ($=1\bmod 3$)
\item $C=\set{2,5,8}$ ($=2 \bmod 3$)
\end{itemize}
Need sum to be zero mod $3$.  If any two are equal $\bmod 3$, then
the third must also be equal.  So they are all same or all
different.  Thus the possibilities are:
\begin{itemize} 
\item 3 from $A$: ${3 \choose 3}=1$.
\item 3 from $B$: ${4 \choose 3} = 4$.
\item 3 from $C$: ${3 \choose 3} = 1$.
\item One from each: $3 \cdot 4 \cdot 3 = 36$ (product rule).
\end{itemize}
Total: 42.
\end{example}


\end{document}
