% Begin your document as any latex file
%\documentclass{article}
% Use package 6895
%\usepackage{../6895}
%\usepackage{epsfig}

% The following are just for the sake of this document.  Ignore them
% for the purposes of generating scribe notes.
% --------------------------------------------------------------------
\newcommand{\pc}[1]{{\tt\begin{description} \item [\TT] #1
\end{description}}}

\newcommand{\com}[1]{$\backslash$#1}
\newcommand{\comone}[2]{$\backslash$#1\{#2\}}
\newcommand{\comtwo}[3]{$\backslash$#1\{#2\}\{#3\}}
\newcommand{\comthree}[4]{$\backslash$#1\{#2\}\{#3\}\{#4\}}
\newcommand{\comfour}[5]{$\backslash$#1\{#2\}\{#3\}\{#4\}\{#5\}}
\newcommand{\comfive}[6]{$\backslash$#1\{#2\}\{#3\}\{#4\}\{#5\}\{#6\}}

\newcommand{\sh}[2]{\com{#1} & $#2$ \\}
\newcommand{\shn}[2]{\com{#1} & #2 \\}

\newenvironment{todolist}{\begin{description}}{\end{description}}
\newcommand{\todoitem}[2]{\item {\em #1} \hfill \\ #2}
% ---------------------------------------------------------------------

% Begin the lecture with a line like the following:
% This replaces the usual \begin{document} that a latex
% file begins with.
\begin{lecture}{2}{Cilk, Matrix Multiplication, and Sorting}{Jason Hickey and Tyeler Quentmeyer}

% Now summarize the lecture.  This should be like an abstract
% for the lecture, broken down into a few bullet points.
\begin{summary}
  % The first item of each \sumitem is the title of the part of the
  % lecture, and the second part is a description (i.e. abstract) of
  % that part.
  \sumitem{Parallel Processing With Cilk}{This section provides a brief introduction to the Cilk language and how Cilk
  schedules and executes parallel processes.}
  \sumitem{Parallel Matrix Multiplication}{This section shows how to
  multiply matrices efficiently in parallel.}
  \sumitem{Parallel Sorting}{This section describes a parallel implementation
  of the merge sort algorithm.}
\end{summary}

% Now organize your lecture into sections.

\section{Parallel Processing With Cilk}

We need a systems background to implement and test our parallel
systems theories.  This section gives an introduction to the Cilk
parallel-programming language.  It then gives some background on
how Cilk schedules parallel processes and examines the role of
race conditions in Cilk.

\subsection{Cilk}

This section introduces Cilk.  Cilk is a version of C that runs in
a parallel-processing environment. It uses the same syntax as C
with the addition of the keywords \texttt{spawn}, \texttt{sync},
and \texttt{cilk}. For example, the Fibonacci function written in
Cilk looks like this:
\begin{verbatim}
    cilk int fib(int n)
    {
       if(n < 2) return n;
       else
        {
            int x, y;
            x = spawn fib(n-1);
            y = spawn fib(n-2);
            sync;
            return(x + y);
        }
    }
\end{verbatim}
Cilk is a \defn{faithful} extension of C, in that if the Cilk
keywords are elided from a Cilk program, the result is a C program
which implements the Cilk semantics.

A function preceded by \texttt{cilk} is defined as a Cilk
function. For example,
\begin{verbatim}
    cilk int fib(int n);
\end{verbatim}
defines a Cilk function called \texttt{fib}.  Functions defined
without the \texttt{cilk} keyword are typical C functions.  A
function call preceded by the \texttt{spawn} keyword tells the
Cilk compiler that the function call can be made asynchronously in
a concurrent thread. The \texttt{sync} keyword forces the current
thread to wait for asynchronous function calls made from the
current context to complete.

Cilk keywords introduce several idiosyncracies into the C syntax.
A Cilk function cannot be called with normal C calling conventions
-- it must be called with \texttt{spawn} and waited for with
\texttt{sync}.  The \texttt{spawn} keyword can only be applied to
a Cilk function.  The \texttt{spawn} keyword cannot occur within
the context of a C function.  Refer to the Cilk manual for more
details.

\subsection{Parallel-Execution Model}

This section examines how Cilk runs processes in parallel.  It
introduces the concepts of work-sharing and work-stealing and then
outlines Cilk's implementation of the work-stealing algorithm.

Cilk processes are scheduled using an online greedy scheduler. The
performance bounds of the online scheduler are close to the
optimal offline scheduler.  We will look at provable bounds on the
performance of the online scheduler later in the term.

Cilk schedules processes using the principle of work-stealing
rather than work-sharing.  Work-sharing is where a thread is
scheduled to run in parallel whenever the runtime makes an
asynchronous function call. Work-stealing, in contrast, is where a
processor looks around for work whenever it becomes idle.

To better explain how Cilk implements work-stealing, let us first
examine the call stack of a vanilla C program running on a single
processor.
\begin{figure}
\centering
\epsfig{file=2/worksteal.ps,%
    height=5cm}
\caption{Call stack of an executing process.  Boxes represent
stack frames.}  \label{fig:Stack}
\end{figure}
In figure \ref{fig:Stack}, the stack grows downward.  Each stack
frame contains local variables for a function.  When a function
makes a function call, a new stack frame is pushed onto the stack
(added to the bottom) and when a function returns, it's stack
frame is popped from the stack.  The call stack maintains
synchronization between procedures and functions that are called.

In the work-sharing scheme, when a function is spawned, the
scheduler runs the spawned thread in parallel with the current
thread.  This has the benefit of maximizing parallelism.
Unfortunately, the cost of setting up new threads is high and
should be avoided.

Work-stealing, on the other hand, only branches execution into
parallel threads when a processor is idle.  This has the benefit
of executing with precisely the amount of parallelism that the
hardware can take advantage of.  It minimizes the number of new
threads that must be setup.  Work-stealing is the lazy way to put
off work for parallel execution until parallelism actually occurs.
It has the benefit of running with the same efficiency as a serial
program in a uniprocessor environment.

Another way to view the distinction between work-stealing and
work-sharing is in terms of how the scheduler walks the
computation graph.  Work-sharing branches as soon and as often as
possible, walking the computation graph with a breadth-first
search.  Work-stealing only branches when necessary, walking the
graph with a depth-first search.

Cilk's implementation of work-stealing avoids running threads that
are likely to share variables by scheduling threads to run from
the other end of the call stack. When a processor is idle, it
chooses a random processor and finds the sleeping stack frame that
is closest to the base of that processor's stack and executes it.
This way, Cilk always parallelizes code execution at the oldest
possible code branch.

\subsection{Cactus Stack}

Cilk uses a \defn{cactus stack} to implement C's rule for sharing
of function-local variables. A cactus stack is a parallel stack
implemented as a tree.  A push maps to following a branch to a
child and a pop maps to returning to the parent in a tree.
\begin{figure}
\centering
\epsfig{file=2/cactus.ps,%
    height=3cm}
\caption{Example of a call stack shown as a cactus stack, and the
views of the stack as seen by each procedure.  Boxes represent
stack frames.}    \label{fig:Cactus}
\end{figure}
For example, the cactus tree in figure \ref{fig:Cactus} represents
the call stack constructed by a call to A in the following code:
\begin{verbatim}
    void A(void)
    {
        B();
        C();
    }

    void B(void)
    {
        D();
        E();
    }

    void C(void)
    {
        F();
    }

    void D(void) {}

    void E(void) {}

    void F(void) {}
\end{verbatim}

Cilk has the same rules for pointers as C.  Pointers to local
variables can be passed downwards in the call stack.  Pointers can
be passed upward only if they reference data stored on the heap
(allocated with \texttt{malloc}).  In other words, a stack frame
can only see data stored in the current and in previous stack
frames. Functions cannot return references to local variables.

The complete call tree is shown in figure \ref{fig:Cactus}. Each
procedure sees a different view of the call stack based on how it
is called. For example, B sees a call stack of A followed by B, D
sees a call stack of A followed by B followed by D and so on. When
procedures are run in parallel by Cilk, the running threads
operate on their view of the call stack.  The stack maintained by
each process is a reference to the actual call stack, not a copy
of it.  Cilk maintains coherence among call stacks that contain
the same frames using methods that we will discuss later.

\subsection{Race Conditions}

The single most prominent reason that parallel computing is not
widely deployed today is because of race conditions.  Identifying
and debugging race conditions in parallel code is hard. Once a
race condition has been found, no methodology currents exists to
write a regression test to ensure that the bug is not reintroduced
during future development.  For these reasons, people do not write
and deploy parallel code unless they absolutely must.  This
section examines an example race condition in Cilk.

Consider the following code:
\begin{verbatim}
    cilk int foo(void)
    {
        int x = 0;
        spawn bar(&x);
        spawn bar(&x);
        sync;
        return x;
    }

    cilk void bar(int *p)
    {
        *p += 1;
    }
\end{verbatim}
If this were a serial code, we would expect that foo returns 2.
What value is returned by foo in the parallel case? Assume the
increment performed by bar is implemented with assembly that looks
like this:
\begin{verbatim}
    read x
    add
    write x
\end{verbatim}
Then, the parallel execution looks like the following:
\begin{verbatim}
    bar 1:
    read x (1)
    add
    write x (2)

    bar 2:
    read x (3)
    add
    write x (4)
\end{verbatim}
where bar 1 and bar 2 run concurrently.  On a single processor,
the steps are executed (1) (2) (3) (4) and foo returns 2 as
expected. In the parallel case, however, the execution could occur
in the order (1) (3) (2) (4), in which case foo would return 1.
The simple code exhibits a race condition.  Cilk has a tool called
the Nondeterminator which can be used to help check for race
conditions.

\section{Matrix Multiplication and Merge Sort}

In this section we explore multithreaded algorithms for matrix
multiplication and array sorting. We also analyze the work and
critical path-lengths.  From these measures, we can compute the
parallelism of the algorithms.

\subsection{Matrix Multiplication}
To multiply two $n \times n$ matrices in parallel, we use a
recursive algorithm.  This algorithm uses the following
formulation, where matrix $A$ multiplies matrix $B$ to produce a
matrix $C$:

\begin{eqnarray*}
\begin{pmatrix} C_{11} & C_{12} \\ C_{21} & C_{22} \end{pmatrix}
&=&
\begin{pmatrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{pmatrix}
\cdot{}
\begin{pmatrix} B_{11} & B_{12} \\ B_{21} & B_{22} \end{pmatrix}
\\
&=&
\begin{pmatrix} A_{11}B_{11}+A_{12}B_{21} & A_{11}B_{12}+A_{12}B_{22} \\ A_{21}B_{11}+A_{22}B_{21} & A_{21}B_{12}+A_{22}B_{22} \end{pmatrix}
.
\end{eqnarray*}

This formulation expresses an $n \times n$ matrix multiplication
as 8 multiplications and 4 additions of $(n/2) \times (n/2)$
submatrices.  The multithreaded algorithm \proc{Mult} performs the
above computation when $n$ is a power of 2. \proc{Mult} uses the
subroutine \proc{Add} to add two $n \times n$ matrices.

\begin{code}
\<$\proc{Mult}(C, A, B, n)$ \\
    \If\ $n = 1$ \\
    \Then\ $C[1,1] \gets A[1,1] \cdot B[1, 1]$ \\
     \Else\ allocate a temporary matrix $T[1\twodots n, 1\twodots n]$ \\
\> \> \>   partition $A, B, C$ and $T$ into $(n/2) \times (n/2)$ submatrices \\
\> \> \>        \Spawn\ $\proc{Mult}(C_{11}, A_{11}, B_{11}, n/2)$ \\
\> \> \>        \Spawn\ $\proc{Mult}(C_{12}, A_{11}, B_{12}, n/2)$ \\
\> \> \>        \Spawn\ $\proc{Mult}(C_{21}, A_{21}, B_{11}, n/2)$ \\
\> \> \>        \Spawn\ $\proc{Mult}(C_{22}, A_{21}, B_{12}, n/2)$ \\
\> \> \>        \Spawn\ $\proc{Mult}(T_{11}, A_{12}, B_{21}, n/2)$ \\
\> \> \>        \Spawn\ $\proc{Mult}(T_{12}, A_{12}, B_{22}, n/2)$ \\
\> \> \>        \Spawn\ $\proc{Mult}(T_{21}, A_{22}, B_{21}, n/2)$ \\
\> \> \>        \Spawn\ $\proc{Mult}(T_{22}, A_{22}, B_{22}, n/2)$ \\
\> \> \>        \Sync \\
\> \> \>        \Spawn\ $\proc{Add}(C, T, n)$ \\
\> \> \>        \Sync \\
\\
\<$\proc{Add}(C, T, n)$ \\
   \If\ $n = 1$ \\
     \Then\ $C[1,1] \gets C[1,1] + T[1,1]$ \\
         \Else\ partition $C$ and $T$ into $(n/2) \times (n/2)$ submatrices \\
\> \> \>            \Spawn\ $\proc{Add}(C_{11}, T_{11}, n/2)$ \\
\> \> \>            \Spawn\ $\proc{Add}(C_{12}, T_{12}, n/2)$ \\
\> \> \>            \Spawn\ $\proc{Add}(C_{21}, T_{21}, n/2)$ \\
\> \> \>            \Spawn\ $\proc{Add}(C_{22}, T_{22}, n/2)$ \\
\> \> \>            \Sync \\
\end{code}

The analysis of the algorithms in this section requires the use of
the Master Theorem.  We state the Master Theorem here for
convenience.

\begin{theorem}[Master Theorem]
Let $a \ge 1$ and $b > 1$ be constants, let $f(n)$ be a function,
and let $T(n)$ be defined on the nonnegative integers by the
recurrence
\begin{eqnarray*}
T(n) = aT(n/b) + f(n) ,
\end{eqnarray*}
where we interpret $n/b$ to mean either $\floor{n/b}$ or
$\ceil{n/b}$.  Then $T(n)$ can be bounded asymptotically as
follows.

\begin{enumerate}
\item If $f(n) = \bigO(n^{\log_b{a-\epsilon}})$ for some constant
$\epsilon > 0$, then $T(n) = \Theta(n^{log_b{a}})$. \item If $f(n)
= \Theta(n^{log_b{a}}lg^k{n})$ for some constant $k \ge 0$, then
$T(n) = \Theta(n^{log_b{a}}lg^{k+1}{n})$. \item If $f(n) =
\Omega(n^{log_b{a+\epsilon}})$ for some constant $\epsilon > 0$,
and if $af(n/b) \le cf(n)$ for some constant $c < 1$ and all
sufficiently large $n$, then $T(n) = \Theta(f(n))$.
\end{enumerate}
\end{theorem}

We begin by analyzing the work for \proc{Mult}.  The work is the
running time of the algorithm on one processor, which we compute
by solving the recurrence relation for the serial equivalent of
the algorithm.  We note that the matrix partitioning in
\proc{Mult} and \proc{Add} takes \bigO(1) time, as it requires
only a constant number of indexing operations.  For the subroutine
\proc{Add}, the work at the top level (denoted $A_1(n)$) then
consists of the work of 4 problems of size $n/2$ plus a constant
factor, which is expressed by the recurrence
\begin{align}
A_1(n) &= 4 A_1(n/2) + \Theta(1) \\
&= \Theta(n^2) .
\end{align}
We solve this recurrence by invoking case 1 of the Master Theorem.
Similarly, the recurrence for the work of \proc{Mult} (denoted
$M_1(n)$):
\begin{align}
M_1(n) &= 8 M_1(n/2) + \Theta(n^2) \\
&= \Theta(n^3) .
\end{align}
 We also solve this recurrence with case 1 of the Master Theorem.  The work is the same as for the traditional triply-nested-loop
serial algorithm.

\begin{figure}
\centering
\epsfig{file=2/squiggle.ps,%
    height=3cm}
\caption{Critical path.  The squiggles represent two different
code paths.  The circle is another code path.}
\label{fig:Squiggle}
\end{figure}

The critical-path length is the maximum path length through a
computation, as illustrated by figure \ref{fig:Squiggle}.  For
\proc{Add}, all subproblems have the same critical-path length,
and all are executed in parallel.  The critical-path length
(denoted $A_\infty(n)$) is a constant plus the critical-path
length of one subproblem, and is represented by the recurrence
(sovled by case 2 of the Master Theorem):
\begin{align}
A_\infty &= A_\infty(n/2) + \Theta(1) \\
&= \Theta(\lg{n}) .
\end{align}
Using this result, the critical-path length for \proc{Mult}
(denoted $M_\infty(n)$) is
\begin{align}
M_\infty &= M_\infty(n/2) + \Theta(\lg{n}) \\
&= \Theta(\lg^2{n}) ,
\end{align}
by case 2 of the Master Theorem.  From the work and critical-path
length, we compute the parallelism:
\begin{align}
M_1(n) / M_\infty(n) = \Theta(n^3/\lg^2{n}) .
\end{align}
As an example, if $n$ = 1000, the parallelism $\approx 10^7$.  In
practice, multiprocessor systems don't have more that $\approx$
64,000 processors, so the algorithm has more than adequate
parallelism.

In fact, it is possible to trade parallelism for an algorithm that
runs faster in practice.  \proc{Mult} may run slower than an
in-place algorithm because of the hierarchical structure of
memory.  We introduce a new algorithm, \proc{Mult-Add}, that
trades parallelism in exchange for eliminating the need for the
temporary matrix $T$.

\begin{code}
\proc{Mult-Add}(C, A, B, n) \\
    \If $n = 1$ \\
\Then $C[1,1] \gets C[1,1] + A[1,1] \cdot B[1, 1]$ \\
    \Else partition $A$, $B$, and $C$ into $(n/2) \times (n/2)$ submatrices \\
\>\>\>          \Spawn $\proc{Mult}(C_{11}, A_{11}, B_{11}, n/2)$ \\
\> \> \>        \Spawn $\proc{Mult}(C_{12}, A_{11}, B_{12}, n/2)$ \\
\> \> \>        \Spawn $\proc{Mult}(C_{21}, A_{21}, B_{11}, n/2)$ \\
\> \> \>        \Spawn $\proc{Mult}(C_{22}, A_{21}, B_{12}, n/2)$ \\
\> \> \>          \Sync \\
\> \> \>        \Spawn $\proc{Mult}(C_{11}, A_{12}, B_{21}, n/2)$ \\
\> \> \>        \Spawn $\proc{Mult}(C_{12}, A_{12}, B_{22} n/2)$ \\
\> \> \>        \Spawn $\proc{Mult}(C_{21}, A_{22}, B_{21}, n/2)$ \\
\> \> \>        \Spawn $\proc{Mult}(C_{22}, A_{22}, B_{22}, n/2)$ \\
\> \> \>        \Sync \\
\end{code}

The work for \proc{Mult-Add} (denoted $M'_1(n)$) is the same as
the work for \proc{Mult}, $M'_1(n) = \Theta(n^3)$.  Since the
algorithm now executes four recursive calls in parallel followed
in series by another four recursive calls in parallel, the
critical-path length (denoted $M'_\infty(n)$) is
\begin{align}
M'_\infty(n) &= 2 M'_\infty(n/2) + \Theta(1) \\
&= \Theta(n)
\end{align}
by case 1 of the Master Theorem. The parallelism is now
\begin{align}
M'_1(n)/M'_\infty = \Theta(n^2) .
\end{align}
When $n$ = 1000, the parallelism $\approx 10^6$, which is still
quite high.

The naive algorithm ($M''$) that computes $n^2$ dot-products in
parallel yields the following theoretical results:
\begin{align}
M''_1(n) &= \Theta(n^3) \\
M''_\infty &= \Theta(\lg{n}) \\
=> Parallelism &= \Theta(n^3/\lg{n}) .
\end{align}
Although it does not use temporary storage, it is slower in
practice due to less memory locality.

\subsection{Sorting}
In this section, we consider a parallel algorithm for sorting an
array.  We start by parallelizing the code for \proc{Merge-Sort}
while using the traditional linear time algorithm \proc{Merge} to
merge the two sorted subarrays.

\begin{code}
\proc{Merge-Sort}(A, p, r) \\
    \If $p < r$ \\
    \Then $q \gets \floor{(p+r)/2}$ \\
\> \> \>        \Spawn $\proc{Merge-Sort}(A, p, q)$ \\
\> \> \>        \Spawn $\proc{Merge-Sort}(A, q+1, r)$ \\
\> \> \>          \Sync \\
\> \> \>        $\proc{Merge}(A, p, q, r)$ \\
\end{code}
Since the running time of \proc{Merge} is $\Theta(n)$, the work
(denoted $T_1(n)$) for \proc{Merge-Sort} is
\begin{align}
T_1(n) &= 2T_1(n/2) + \Theta(n) \\
&= \Theta(n \lg n)
\end{align}
by case 2 of the Master Theorem. The critical-path length (denoted
$T_\infty(n)$) is the critical-path length of one of the two
recursive spawns plus that of \proc{Merge}:
\begin{align}
T_\infty(n) &= T_\infty(n/2) + \Theta(n) \\
&= \Theta(n)
\end{align}
by case 3 of the Master Theorem. The parallelism,
$T_1(n)/T_\infty$ = $\Theta(\lg n)$, is not scalable.  The
bottleneck is the linear-time \proc{Merge}. We achieve better
parallelism by designing a parallel version of \proc{Merge}.

\begin{code}
\proc{P-Merge}(A[1..l], B[1..m], C[1..n]) \\
    \If $m > l$ \\
        \Then \Spawn $\proc{P-Merge}(B[1\twodots m], A[1\twodots l], C[1\twodots n])$ \\
    \ElseIf\ $n = 1$ \\
         \Then $C[1] \gets A[1]$ \\
    \ElseIf\ $l = 1$ \\
         \Then \If $A[1] \leq B[1]$ \\
\>\>            \Then\ $C[1] \gets A[1]; C[2] \gets B[1]$ \\
\>\>            \Else\ $C[1] \gets B[1]; C[2] \gets A[1]$ \\
        \Else find $j$ such that $B[j] \leq A[l/2] \leq B[j+1]$ using binary search \\
\>\>            \Spawn $\proc{P-Merge}(A[1..(l/2)], B[1..j], C[1..(l/2 + j)])$ \\
\>\>            \Spawn $\proc{P-Merge}(A[(l/2 + 1)..l], B[(j+1)..m], C[(l/2 + j + 1)..n])$ \\
\>    \Sync
\end{code}

\proc{P-Merge} puts the elements of arrays $A$ and $B$ into array
$C$ in sequential order, where $n$ = $l$ + $m$.  The algorithm
finds the median of the larger array and uses it to partition the
smaller array.  Then, it recursively merges the lower portions and
the upper portions of the arrays.  The operation of the algorithm
is illustrated in figure \ref{fig:Merge}.
\begin{figure}
\centering
\epsfig{file=2/pmerge.ps,%
    height=3cm}
\caption{Find where middle element of A goes into B.  The boxes
represent arrays.} \label{fig:Merge}
\end{figure}

We begin by analyzing the critical-path length of \proc{P-Merge}.
The critical-path length is equal to the maximum critical-path
length of the two spawned subproblems plus the work of the binary
search. The binary search completes in $\Theta(\lg m)$ time, which
is $\Theta(\lg n)$ in the worst case.  For the subproblems, half
of $A$ is merged with all of $B$ in the worst case.  Since $l \ge
n/2$, at least $n/4$ elements are merged in the smaller
subproblem.  That leaves at most $3n/4$ elements to be merged in
the larger subproblem.  Therefore, the critical-path is
\begin{eqnarray*}
T_\infty(n) &\leq T(3/4n) + O(\lg{n}) \\
&= O(\lg^2{n})
\end{eqnarray*}
by case 2 of the Master Theorem. To analyze the work of
\proc{P-Merge}, we set up a recurrence by using the observation
that each subproblem operates on $\alpha n$ elements, where $1/4
\le \alpha \le 3/4$.  Thus, the work satisfies the recurrence
\begin{eqnarray*}
T_1(n) &= T(\alpha n) + T((1 - \alpha) n) + O(\lg{n}) .
\end{eqnarray*}
We shall show that $T_1(n) = \Theta(n)$ by using the substitution
method.  We take $T(n) \le an - b \lg n$  as our inductive
assumption, for constants $a,b > 0$.  We have
\begin{eqnarray*}
T_(n) &\le& a \alpha n - b \lg(\alpha n) + a(1 - \alpha)n - b \lg((1 - \alpha) n) + \Theta(\lg n) \\
&=& an - b(\lg(\alpha n) + \lg((1 - \alpha)n)) + \Theta(\lg n) \\
&=& an - b(\lg \alpha + \lg n + \lg(1 - \alpha) + \lg n) + \Theta(\lg n) \\
&=& an - b \lg n - (b(\lg n + \lg(\alpha(1 - \alpha))) - \Theta(\lg n)) \\
&\le& an - b \lg n ,
\end{eqnarray*}
since we can choose $b$ large enough so that $b(\lg n +
\lg(\alpha(1-\alpha)))$ dominates $\Theta(\lg n)$.  We can also
pick $a$ large enough to satisfy the base conditions.  Thus,
$T_(n) = \Theta(n)$, which is the same as the work for the
ordinary \proc{Merge}. Reanalyzing the \proc{Merge-Sort}
algorithm, with \proc{P-Merge} replacing \proc{Merge}, we find
that the work remains the same, but the critical-path length is
now
\begin{align}
T_\infty(n) &= T_\infty(n/2) + \Theta(\lg^2{n}) \\
&= \Theta(\lg^3{n})
\end{align}
by case 2 of the Master Theorem. The parallelism is now $\Theta(n
\lg n)/\Theta(\lg^3{n})$ = $\Theta(n/\lg^2{n})$.  By using a more
clever algorithm, a parallelism of $\Omega(n/\lg{n})$ can be
achieved.

While it is important to analyze the theoritical bounds of
algorithms, it is also necessary that the algorithms perform well
in practice.  One short-coming of \proc{Merge-Sort} is that it is
not in-place.  An in-place parallel version of \proc{Quick-Sort}
exists, which performs better than \proc{Merge-Sort} in practice.

Additionally, while we desire a large parallelism, it is good
practice to design algorithms that scale down as well as up.  We
want the performance of our parallel algorithms when running on
one processor to compare well with the performance of the serial
version of the algorithm.  The best sorting algorithm to date
requires only \percent{20} more work than the serial equivalent.
Coming up with a dynamic multithreaded algorithm which works well
in practice is a good research project.

\end{lecture}
\theend
