\documentstyle[12pt]{article}
\input{macros-course}
\begin{document}
\section{The Poisson Distribution}

Let's return to the Birthday problem we studied last a while ago.  We
assumed that each person in the room had a birthday chosen
independently at random from among the days of the year.  It followed
that with $n$ people in the room, the number of people with birthdays
on a given day had a binomial distribution with parameters $n$ (number
of people) and $p=1/365$ as the probability a particular person has a
birthday on a particular day.  Now $n$ is large and $p$ is small while
$\mu=np$ (the expected number of people with birthdays in a given
day) is medium sized.  There is a useful approximation to the binomial
distribution in this case.  Let's first estimate the probability $p_0$
of no birthdays on a given day:
\begin{eqnarray*}
p_0 &= (1-p)^n\\
  &\approx e^{-np}\\
  &=e^{-\lambda}
\end{eqnarray*}

Now let $p_k$ be the probability of exactly $k$ people.  Recall that
\begin{eqnarray*}
\frac{p_{k+1}}{p_k} &=
\frac{{n\choose{k+1}}p^{k+1}(1-p)^{n-k-1}}{{n\choose k}p^k(1-p)^{n-k}}\\
&= \frac{(n-k)p}{k(1-p)}\\
&= \frac{\mu-kp}{k(1-p)}\\
&\approx \frac{\mu}{k},
\end{eqnarray*}
where the approximation assumes that $kp \ll \mu$ and $p \ll 1$.  It
follows that $p_1=p_0\mu$, $p_2=p_0\mu^2/2$, and in general,
\[
p_k = e^{-\mu}\frac{\mu^k}{k!}.
\]

Therefore, for the approximately 100 people in this room, we have
$\mu=.27397$.  It follows that
\begin{eqnarray*}
p_0 &\approx .76\\
p_1 &\approx .20831\\
p_2 &\approx .02\\
p_3 &\approx .002
\end{eqnarray*}

It turns out that
\begin{eqnarray*}
\sum p_k &= \sum e^{-\mu} \frac{\mu^k}{k!}\\
         &= e^{-\mu} \sum \frac{\mu^k}{k!}\\
         &= 1
\end{eqnarray*}

So we can see $p_k$ as an actual probability distribution.  Is there
an experiment that this distribution describes?    Well, suppose that
we consider some lengthy period of time $T$ over which $n$ events happen
at random times.  The ``rate'' at which events happen is
$\lambda=n/T$.  The probability that any one of these events happens
in any interval of length $t$ is $t/T$.  Thus the expected number of
events in the interval is $nt/T = \lambda t$.  This leads to the
following distribution: over any interval of time time, the
probability of exactly $k$ events is
\[
p_ = e^{-\lambda t}\frac{(\lambda t)^k}{k!}
\]



There is actually a
small set of axioms which leads to the Poisson distribution.  Let us
suppose that people are being born independently at random with rate
$\lambda$.  That is, the average number of people born in a time interval
of length $t$ is $\lambda t$.  Some obvious axioms about this ``birth
process'' are the following:
\begin{itemize}
\item The number of births in any given time interval is independent of the
number of births outside that interval.
\item In any interval of length $h$, the expected number of births is
  $\lambda h$.
\end{itemize}





\section{Higher order statistics}

\begin{defin} $E[R^m] $ is the m-th moment of R. \end{defin}
Thus the first moment is the mean, the second one is the variance.
The more moments  we know, the more we know the distribution of R. In fact,
if you have all moments for some $R:S\rightarrow N$,
 then you can recover the underlying
distribution function for R. The method uses generating functions.
\begin{defin}
The moment generating function for R is
\[H_R(x)=\sum_n \frac{E[R^n]}{n!}x^n .\] \end{defin}
\begin{defin}
The probability  generating function for R is 
\[G_R(x)=\sum_nPr(R=n)x^n .\] \end{defin}

These generating functions are very closely related as the following theorem
shows: 
\begin{theorem}
If $R:S\rightarrow N$, then
\[H_R(t)=G_R(e^t).\] \end{theorem}
\begin{proof} 
\[H_R(t)=\sum_i \frac{E[R^i]}{i!}t^i=\sum_i \frac{t^i}{i!}\sum_n n^i Pr(R=n)
=\] 
\[=\sum_n Pr(R=n)\sum_i\frac{t^in^i}{i!}= \sum_nPr(R=n)e^{tn}=\]
\[=\sum_n Pr(R=n)(e^t)^n = G_R(e^t). \] 
\end{proof}


Going from G to H is similar to computing Laplace our Fourier transforms.
\begin{defin} The exponential transform of R is $Ex(e^{tr})$. \end{defin}
\end{document} 
% Local Variables: 
% mode: latex
% TeX-master: t
% End: 
