% Lecture notes - Lecture 1 - 09/03/2003
% Abhi Shelat and Ben Adida
%\documentclass{article}
% Use package 6895
%\usepackage{../6895}
\input epsf

\begin{lecture}{1}{Introduction \& Dynamic Multithreading}{Abhi Shelat and Ben Adida}

\begin{summary}
  \sumitem{Course Overview}{We present an overview of the course topics.}
  \sumitem{Introduction to Parallel Programs}{We define basic terminology and present our first parallel program.}
  \sumitem{Modeling and Measuring Parallel Programs}{We define the graph model for parallel programming and introduce a methodology for measuring the performance of parallel progams. }
  \sumitem{Greedy Scheduler}{We prove that a simple greedy scheduler executes in no worse than twice the theoretically optimal schedule.}
  \sumitem{An Anecode}{We review the results of the $\star$Socrates to show how empirical performance analysis does not suffice in understanding parallel computation.}
\end{summary}

\section{Course Overview}
\seclabel{course-overview}

This course {\bf 6.895}  will focus on theoretical underpinnings of
parallel computation, though it is clear that theory doesn't explain
everything. This year, the course will address topics in a top-down
fashion so as to help students pick final projects early on:
\begin{enumerate}
\item High-level models
  \begin{enumerate}
  \item multithreading
  \item scheduling and load-balancing
  \item data-race detection
  \item synchronization (locking and lock-free algorithms)
  \end{enumerate}
\item Network architecture
  \begin{enumerate}
  \item memory consistency
  \item routing networks (cf. RAW project, configurable networks)
  \end{enumerate}
\item VLSI theory
  \begin{enumerate}
  \item graph layouts
  \item area/volume
  \item $AT^2$ complexity
  \item arithmetic circuits
  \item universality (routing networks and VLSI's version of the
    Universal Turing Machine)
  \end{enumerate}
\end{enumerate}

\section{Introduction to Parallel Programs}
\seclabel{introduction}

Consider the following program to compute the $n$th Fibonacci number.
This program is particularly bad: it runs in exponential time where a
straight-forward iterative method yields linear time and a clever
approach based on repeated squarings of a particular matrix yields
$O(\log(n))$ time [see Cormen, Leiserson, Rivest page 902].
Nonetheless, this program is a good example for introducing our model
of parallel computation.

\begin{verbatim}
fib(n)
    if n < 2
    then return n
    
    x <- spawn fib(n-1)
    y <- spawn fib(n-2)
    
    sync
    return (x+y)
\end{verbatim}


A \texttt{spawn} instruction allows the next instruction of the parent program
to be executed at the same time as the child, which in Unix programming,
is equivalent to a \texttt{fork}. A \texttt{sync}
instruction requires that all locally spawned children finish their
work before proceeding with the computation. In Unix programming, a
\texttt{sync} is equivalent to a \texttt{join}.

Our syntax for the presented program describes \defn{logical parallelism}. We
do not explicitly state which tasks are assigned to which processors.
Instead, we focus on the computations which can be safely performed in
parallel. It is the \defn{scheduler}'s job to determine how to map
dynamically unfolding execution onto a set of processors.

\section{Modeling and Measuring Parallel Programs}
\seclabel{modeling-and-measuring}

\subsection{Instructions Streams as a DAG}

\begin{figure}
  \epsfbox{1/l1.eps}
\caption{The instruction stream for \texttt{fib(4)}.  Each circle
  represents a thread.  Each solid arrow represents a \texttt{continuation},
  each dashed arrow represents a \texttt{spawn} operation, and each
  dotted arrow represents a \texttt{return} from a spawned procedure.}
\end{figure}

We model the instruction stream of a program as a {\bf computation directed acyclic graph
  (DAG)}. Figure 1 shows the \defn{computation DAG} for \texttt{fib(4)}.
  Vertices are \defn{threads} which is a maximal
  sequence of instructions not containing parallel control (\texttt{spawn},
   \texttt{return} from a spawned procedure, or \texttt{sync}).

We have three threads:
\begin{itemize}
\item Thread A: The \texttt{if} statement up to the first \texttt{spawn}.
\item Thread B: The computation of \texttt{n-2} before the second \texttt{spawn}.
\item Thread C: The \texttt{x+y} operation before the \texttt{return}.
\end{itemize}

\subsection{Performance Measures}

We define several performance measures to evaluate various
scheduler algorithms. We assume an ideal situation: no cache issues,
no interprocessor communication costs. These results may not hold
under different assumptions, but we are after the essence of understanding.

\begin{itemize}
\item $T_P$ is the minimum running time on $P$ processors
\item $T_1$, also called \defn{work}, is the minimum running time on 1 processor
\item $T_\infty$ is the minimum running time on an infinite number of processors.  This is equivalent to the longest 
path in the DAG and is referred to as the \defn{critical path length}.
\end{itemize}

\noindent In our example, if all threads run in unit time, we have $T_1 = 17$. In practice, the threads would be weighted by their individual runtimes. The \defn{critical path length} is $T_\infty = 8$.

These measures immediately define lower bounds:

\begin{equation}
  \label{lower-bound-tp-1}
  T_P \ge T_1 / P
\end{equation}
\begin{proof-idea}
  Assume by contradiction that $T_P < T_1 / P$. In the best case, $P$
  processors can do at most $P$ work in 1 step.  By multiplying both
  sides by $P$, then $PT_P < T_1$, which implies that the total number
  of operations performed (left-hand side) is less than $T_1$, a
  contradiction.
\end{proof-idea}

\begin{equation}
  \label{lower-bound-tp-2}
  T_P \ge T_\infty
\end{equation}
\begin{proof-idea}
  Once again, suppose not. If $T_P < T_\infty$, simply using $P$ of
  the unlimited number of processors available reduces $T_\infty$,
  which is a contradiction.
\end{proof-idea}
  
\begin{equation}
  \label{speedup}
  T_1/T_P = \mbox{{\bf speedup}}
\end{equation}
\begin{proof-idea}
  With $P$ processors, the maximum speedup is $P$. (Remember: the
  theory only applies to this simplified model.)
\end{proof-idea}

A program's parallel execution plan can have:
\begin{itemize}
\item linear speedup: $T_1/T_P = \Theta(P)$
\item superlinear speedup: $T_1/T_P = \omega(P)$ (not possible in this
  model, though it is possible in others).
\item sublinear speedup: $T_1/T_P = o(p)$
\end{itemize}

The maximum possible speedup for a program given particular $T_1$ and
$T_\infty$ is $T_1/T_\infty$, also called \defn{parallelism}. The
\defn{parallelism} is the average amount of work that can be done in
parallel for every step along the critical path.

\section{The Greedy Scheduler}
\seclabel{greedy-scheduler}

A scheduler's job is to map a computation to particular processors.
In reality, a scheduler typically must make decisions about what tasks
or threads to perform on particular processors without the benefit of
foresight, i.e, it does not know what it must schedule in the future.
In general, these online schedulers are quite complex and will be
discussed later in the term.  Therefore, to begin our analysis, we
focus on offline scheduling. Specifically, we consider the {\defn Greedy
  Scheduler}, a scheduler which attempts to do as much work as
possible at every step.

In any schedule, there are two types of steps:
\begin{itemize}
\item {\bf complete step}: There are at least $P$ threads that are
  ready to run. The greedy scheduler selects any $P$ of them and runs
  them.
\item {\bf incomplete step}: There are strictly less than $P$ threads
  that are ready to run. The greedy scheduler runs them all.
\end{itemize}

Our first important theorem of the course is a beautiful argument, which
extolls the virtue of greedy scheduling.

\begin{theorem}
  \label{greedy-theorem}
  (Graham, Brent) Given $P$ processors, a greedy scheduler executes
  any computation $G$ with work $T_1$ and critical path length
  $T_\infty$ in
  time: \\
  \begin{eqnarray}
    T_p &\le& T_1/p + T_\infty
  \end{eqnarray}
\end{theorem}

\begin{proof}
  In any schedule, there are only two types of steps: complete, and
  incomplete.
  
  We first consider the complete steps. There can be no more than $T_1/p$
  complete steps; otherwise, $\mbox{work} > T_1$, which contradicts the
  definition of $T_1$.
  
  We now consider the incomplete steps. Let $G'$ be the subgraph of
  $G$ that remains to be executed immediately prior to a given
  incomplete step, as shown in Figure 2.  We note that all threads
  with in-degree 0 in $G'$ are ready to be executed. Since we are
  dealing solely with incomplete steps, we also know that all of these
  threads can be immediately executed. The critical path of $G'$
  starts with a thread of in-degree 0. Thus, every incomplete step
  reduces the critical-path length by one unit. In other words, there
  can be no more than $T_\infty$ incomplete steps.

  \begin{figure}
    \epsfbox{1/l2.eps}
    \caption{The leftover graph $G'$ to execute during an incomplete step. Each nodes represents threads. The vertices represent continuations from one thread to another (due to parallel control such as \texttt{spawn} or \texttt{continue}). The nodes marked with an 'X' are threads already executed.}
  \end{figure}
  
  Thus, the total time for executing incomplete and complete steps in a
  greedy scheduler is at most $T_1/P + T_\infty$.
\end{proof}

\begin{corollary}
  A greedy scheduler is always within a factor of 2 of optimal
\end{corollary}

\begin{proof}
  Given the lower bounds \ref{lower-bound-tp-1} and
  \ref{lower-bound-tp-2}, we can express a more concise lower bound as:
  
  \begin{equation}
    \label{concise-lower-bound-tp}
    T_P \ge \max(T_1/P,T_\infty)
  \end{equation}

  In addition, we can trivially express:

  \begin{equation}
    T_1/P \le \max(T_1/P,T_\infty)
  \end{equation}
  \begin{equation}
    T_\infty \le \max(T_1/P,T_\infty)
  \end{equation}

  Given Theorem \ref{greedy-theorem}, we conclude

  \begin{eqnarray}
    T_P &\le& T_1/P + T_\infty \\
    &\le& \max(T_1/P, T_\infty) + \max(T_1/P,T_\infty) \\
    &\le& 2 \max(T_1/P, T_\infty)
  \end{eqnarray}
  
  which shows that $T_P$ is within twice lower bound \ref{concise-lower-bound-tp}.
\end{proof}

  
Every step either contributes to progress on the work (complete
step), or progress on the critical path (incomplete step). A greedy
scheduler is generally a good scheduler.


\begin{corollary}
  The greedy scheduler achieves linear speedup when $P = O(T_1/T_\infty)$
\end{corollary}

\begin{proof}
  \begin{eqnarray}
    T_p 
    &\le& T_1/p + T_\infty \\
    &=& T_1/p + O(T_1/p) \\
    &=& \Theta(T_1/p)
  \end{eqnarray}
\end{proof}

The idea is to operate in the range where $T_1/P$ dominates
$T_\infty$. Given a particular problem with fixed $T_1$ and
$T_\infty$, adding more processors increases $P$ and reduces $T_1/P$
while $T_\infty$ remains unaffected. Once $T_\infty$ becomes
significant in comparison to $T_1/P$, the critical path's dependencies
reduce the ability to parallelize the computation to additional
processors. As long as $T_1/P$ dominates $T_\infty$, all processors
can be used efficiently.

\section{Anecdote}

$\star$Socrates, a chess program, had two versions, one including an
optimization that significantly improved performance on 32
processors. The performance improvement reduced the total work, but
increased the critical-path length:

\begin{itemize}
\item $T_{32} = 65 sec$, $T_1 = 2048$, $T_\infty = 1$
\item $T'_{32} = 40 sec$, $T'_1 = 1024$, $T_\infty = 8$
\end{itemize}

On 512 processors, however, the optimization becomes a problem:

\begin{itemize}
\item $T_{512} = T_1 / 512 + T_\infty = 5$
\item $T'_{512} = T'_1 / 512 + T'_\infty = 10$
\end{itemize}

The moral of the story: work and critical-path length are more important to understand thean simple empirical performance measurements.

\end{lecture}
\theend
